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1.  Introduction 


1.1  Background 

This  is  the  third  in  a  series  of  experiments  designed  to  investigate  how  best  to  scale  robot 
controls  and  displays  for  dismounted  Soldiers  who  need  smaller  and  lighter  devices.  The  first 
two  experiments  in  this  series  addressed  screen  size  for  the  dismounted  Soldier’s  driving  camera 
display  (Redden,  Pettitt,  Carstens,  and  Elliott,  2008)  and  controller  options  for  the  dismounted 
Soldier  to  drive  the  robot  and  maneuver  the  robotic  arm  (Pettitt,  Redden,  Carstens,  and  Elliott, 
2008).  The  environments  of  dismounted  Soldiers  are  rugged  and  physically  demanding,  and 
Soldiers  must  carry  their  robotic  operator  control  units  (OCU)  along  with  all  their  protective  and 
fighting  equipment  in  these  challenging  environments.  Relatively  large  displays  typically  used 
in  a  stationary  environment  or  inside  a  combat  vehicle  are  not  appropriate  and  could  have  an 
adverse  impact  on  the  dismounted  Soldiers’  missions.  Scaling  robotic  interfaces  involves  the 
design  and  development  of  smaller,  lighter  versions  that  are  still  rugged,  easy  to  use,  easy  to 
learn,  and  easy  to  maintain.  Scaling  ensures  that  training  transfer  is  easy  across  environments 
and  that  interfaces  are  tailored  to  the  environment  in  which  they  are  used.  The  key  to  successful 
scaling  is  to  consider  the  range  of  devices  that  Soldiers  will  use  (e.g.,  vehicle-mounted  robot 
control  devices,  other  controller  devices)  and  also  their  context  of  use.  A  smaller  controller  may 
be  easy  to  learn  and  use  if  it  is  similar  to  existing  controllers.  On  the  other  hand,  some  controller 
characteristics  will  not  be  as  effective  in  a  smaller  unit.  Consider  the  increased  difficulty  of 
typing  on  a  QWERTY  keyboard  on  a  cell  phone  compared  to  a  computer  keyboard.  Context  of 
use  also  becomes  a  factor,  when  Soldiers  must  use  their  displays  in  rough  terrain,  in  bright 
daylight,  or  perhaps,  while  on  the  move.  Trade-offs  in  controller  options  for  different  task 
demands  must  be  recognized  and  considered.  Ultimately,  scaling  depends  on  user  evaluations 
and  experimental  controlled  investigations  under  realistic  task  demands. 

Several  past  studies  have  found  no  performance  differences  created  by  reduced  display  size 
(Alexander,  Wickens,  and  Hardy,  2005;  Minkov,  Perry  and  Oron-Gilad,  2007;  Muthard  and 
Wickens,  2004;  Redden  et  al.  2008;  Stark,  Comstock,  Prinzel,  Burdette,  and  Scerbo,  2001; 
Stelzer  and  Wickens,  2006).  However,  there  were  conflicting  results  with  some  investigations 
indicating  a  benefit  of  larger  displays.  In  addition,  previous  experiments  were  conducted  for  a 
variety  of  task  demands,  some  of  which  were  not  directly  generalizable  to  the  dismounted  robot 
control  situation.  Redden  et  al.  (2008)  conducted  an  experiment  to  evaluate  display  size  options 
in  a  manner  that  maximizes  external  generalizability — experienced  Soldiers  used  real  control 
unit  options  to  accomplish  typical  dismount  controller  tasks.  That  study  indicated  that  smaller 
displays  (i.e.,  3.5-in.  and  6.5-in.  diagonal  displays)  were  as  effective  for  teleoperation  and  local 
surveillance  with  small,  slower  speed  robots  as  a  10.5  in.  diagonal  display  and  a  goggle-mounted 
display.  The  finding  that  the  camera  display  can  be  smaller  allows  more  screen  real  estate  to  be 
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made  available  for  other  purposes  (i.e.,  map  displays,  menus,  etc.)  and  enables  the  use  of  lower 
resolution  cameras,  which  would  allow  faster  video  processing  with  less  bandwidth  requirement. 
It  is  important  to  note  that  the  findings  of  this  experiment  were  limited  to  camera  displays  for 
driving  small  robots  and  performing  local  surveillance;  they  did  not  address  other  tasks  or  other 
types  of  displays  (e.g.,  map  displays,  touch  screens,  etc.).  Additional  tasks  may  require  different, 
additional,  or  larger  displays. 

This  experiment  builds  upon  the  Redden  et  al.  (2008)  display  size  experiment  by  addressing  an 
additional  task  that  is  fundamental  to  dismounted  Soldiers — that  of  navigation.  Many  existing 
robot  control  units  provide  a  combination  of  a  camera  display  and  a  map  display  to  enable 
operators  to  navigate  to  locations  that  are  out  of  line-of-sight.  These  displays  are  typically  split 
screen  displays.  In  this  study,  we  compare  the  split  screen  display  option  with  the  option  of 
toggling  between  the  camera  and  map  displays,  which  requires  less  screen  size.  We  also  explore 
the  utility  of  tactile  direction  cues  to  support  navigation  and  obstacle  avoidance  when  using  the 
camera  view.  Tactile  cues  may  minimize  the  need  for  toggling  between  screens  and  enable  the 
operator  to  maintain  situation  awareness  (SA)  and  facilitate  navigation  around  obstacles. 

1.1.1  Display  Size 

The  finding  that  a  3. 5 -in.  diagonal  display  is  as  effective  as  larger  displays  for  driving  a  small 
robot  and  for  detection  of  threats  in  close  proximity  to  the  robot  is  appealing  because  smaller 
display  sizes  can  potentially  reduce  the  overall  OCU  size  requirement  for  control  of  unmanned 
vehicles  as  well  as  the  weight  of  this  equipment.  In  addition,  more  space  on  the  OCU  can  be 
allocated  for  displaying  menus,  maps,  and  representations  of  vehicle  status.  Although  a  3.5-in. 
diagonal  display  was  found  to  be  sufficient  for  remote  driving  with  a  small  robot,  it  may  not  be 
adequate  for  navigating  a  small  robot  using  a  map  display  because  display  size  has  been  found  to 
affect  performance  and  workload  differently,  depending  on  the  task  and  the  conditions.  For 
example,  research  with  pilots  (Wickens,  Muthard,  Alexander,  van  Olffen,  and  Podczerwinski, 
2003)  found  that  display  size  had  no  impact  on  overall  surveillance,  change  detection,  tracking, 
or  response  time  when  using  complex  and  dynamic  aerial  map  displays  with  icons.  In  a 
subsequent  study  using  a  low-fidelity  two-dimensional  (2-D)  and  three-dimensional  (3-D) 
tracking  task,  Stelzer  and  Wickens  (2006)  found  that  performance  was  degraded  when  urgency 
was  high  and  displays  were  smaller.  They  also  compared  effects  of  display  size  during 
surveillance  and  target  search  tasks  using  complex  and  dynamic  aerial  map  display  icons.  They 
found  that  smaller  displays  were  associated  with  a  higher  rate  of  error  in  heading  and  altitude,  as 
well  as  increased  deviations  from  flight  path.  However,  display  size  did  not  affect  response  time 
or  ability  to  estimate  distance  between  aircraft,  detect  changes,  or  maintain  target  distance  from 
the  lead  aircraft.  These  findings  indicate  the  need  to  determine  whether  smaller  displays  can  be 
used  effectively  by  dismounted  Soldiers  performing  a  variety  of  tasks  such  as  land  navigation. 
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1.1.2  Navigation  Displays 

Optimal  display  design  may  depend  on  whether  specific  task  groupings  are  performed 
simultaneously/near-simultaneously  or  sequentially.  It  is  unclear  whether  driving  (e.g.,  camera- 
based  steering,  control,  and  obstacle  avoidance)  and  navigating  (e.g.,  map-based  progress  from 
waypoint  to  waypoint)  are  tasks  that  are  very  closely  coupled  or  if  they  can  be  performed  in  an 
interactive  but  sequential  manner.  Perceptual  and  cognitive  requirements  associated  with  these 
tasks  have  an  impact  on  display  design.  If  the  tasks  can  be  perfonned  sequentially,  then  the 
same  display  real  estate  can  be  used  for  both  tasks  by  having  the  operator  toggle  between  driving 
camera  and  map  displays.  If  the  tasks  can  be  performed  simultaneously  or  even  near 
simultaneously,  then  two  different  displays  would  likely  make  driving  and  navigating  more 
efficient.  Minkov,  Perry,  and  Oron-Gilan  (2007)  used  split  screen  displays  for  unmanned  aerial 
vehicle  (UAV)  navigation  and  monitoring.  However,  the  operators  in  this  experiment  only 
monitored  the  UAV  mounted  camera  while  they  navigated  and  they  did  not  actually  fly/drive  the 
UAV.  If  both  driving  and  navigation  displays  must  be  provided  together,  and  the  driving  display 
is  3.5  in.  in  diagonal,  then  simultaneous  display  of  the  driving  camera  information  and 
navigational  map  information  will  require  display  space  larger  than  3.5  in. 

While  the  toggle  option  would  allow  for  a  smaller  size  display,  it  is  likely  that  Soldiers  will 
experience  higher  manual  and  cognitive  workload.  If  the  task  is  very  dynamic,  such  as  when 
ascertaining  and  avoiding  obstacles,  Soldiers  will  have  to  toggle  back  and  forth  frequently  to 
adjust  their  position  while  keeping  visual  SA  through  the  camera  display.  During  the  task,  the 
Soldier  must  keep  the  alternate  screen  infonnation  in  his  visual  memory.  In  contrast,  the  split 
screen  allows  the  Soldier  to  process  peripheral  visual  information,  thus  allowing  multi-tasking 
with  reduced  workload  (Wickens,  2002).  This  leads  to  our  first  three  hypotheses: 

•  HO  1 .  Soldiers  will  perform  the  robot  navigation  task  more  quickly  with  the  split  screen 
than  with  the  toggle  display. 

•  H02.  Soldiers  will  perform  the  robot  navigation  task  with  fewer  errors  with  the  split 
screen  than  with  the  toggle  display. 

•  H03.  Soldiers  will  identify  hand  signals  more  quickly  and  accurately  with  the  split  screen 
than  with  the  toggle  display. 

An  alternative  approach  for  providing  two  displays  would  be  to  supply  a  visual  display  for 
driving  and  reconnaissance,  and  a  tactile  display  to  support  navigation.  This  would  enable  the 
reduction  of  screen  size  to  3.5  in.  while  providing  both  navigation  and  camera  infonnation. 
According  to  Multiple  Resource  Theory  (MRT),  this  approach  could  reduce  the  burden  on  the 
visual  channel  and  could  be  even  more  efficient  than  providing  two  visual  displays  via  a  split 
screen  (Wickens,  2002),  depending  on  task  demands  and  overall  workload.  Also,  torso-mounted 
tactile  arrays  have  been  used  successfully  for  personal  land  navigation  (Dorneich,  Ververs, 
Whitlow,  and  Mathan,  2006;  Duistermaat,  Elliott,  van  Erp,  and  Redden,  2007;  Elliott,  Redden, 
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Pettitt,  Carstens,  van  Erp,  and  Duistennaat,  2006;  Gilson,  Redden  and  Elliott,  2007;  Van  Erp, 
2007,  2005).  Elliott,  Duistennaat,  Redden,  and  van  Erp  (2007)  found  that  Soldiers  using  a 
helmet-mounted  display  to  navigate  at  night  simultaneously  with  cross-country  movement  found 
fewer  enemy  targets  and  took  longer  to  navigate  the  terrain  than  Soldiers  who  used  a  tactile 
navigation  device  while  moving  cross-country.  Displaying  navigation  information  to  a  different 
information  channel  allowed  simultaneous  navigation  and  cross-country  movement  without 
visual  overload.  In  this  study  condition,  torso-mounted  tactile  cues  would  provide  direction 
guidance  toward  the  next  waypoint  in  a  manner  that  enables  the  Soldier  to  head  toward  the  next 
waypoint,  while  using  the  camera  display  to  drive  the  robot  and  negotiate  obstacles.  Because  the 
information  is  presented  simultaneously,  we  expect  the  multimodal  display  to  be  as  effective  as 
the  split  screen  and  more  effective  than  the  toggle  screen: 

•  H04.  Soldiers  will  perform  the  robot  navigation  task  more  quickly  with  the  multimodal 
display  than  with  the  toggle  display.  There  will  be  no  difference  in  navigation  times 
between  the  multimodal  and  the  split  screen  displays. 

•  H05.  Soldiers  will  perform  the  robot  navigation  task  with  fewer  errors  with  the 
multimodal  display  than  with  the  toggle  display.  There  will  be  no  difference  in  Soldier 
driving  performance  between  the  multimodal  and  the  split  screen  displays. 

•  H06.  Soldiers  will  identify  hand  signals  more  quickly  and  accurately  with  the  multimodal 
display  than  with  the  toggle  display.  There  will  be  no  difference  in  Soldier  performance 
between  the  multimodal  and  the  split  screen  displays. 

1.2  Overview  of  Experiment 

This  study  investigated  the  effect  of  visual  and  multimodal  display  options  on  driving, 
performing  local  surveillance,  and  navigating  a  small  robotic  vehicle.  It  took  place  at  Fort 
Benning,  GA,  and  used  Soldiers  from  the  Officer  Candidate  School  (OCS).  After  training  on  the 
operation  of  the  TALON  system,  each  Soldier  completed  exercises  using  three  different  display 
concepts.  The  first  concept  was  a  6.5-in.  split  screen  display  that  presented  camera-based 
driving  information  on  the  top  portion  of  the  display  and  the  map  display  on  the  bottom.  The 
second  concept  consisted  of  a  3. 5 -in.  display  that  required  the  Soldier  to  toggle  between  the 
driving  camera  and  map  displays.  The  third  concept  was  a  3.5-in.  display  that  was  used 
primarily  for  driving,  while  directional  infonnation  was  provided  to  the  Soldier  via  a  tactile  belt. 
If  the  Soldier  needed  to  see  the  map  display  to  find  distance  information  or  landmark 
information,  he  could  toggle  the  visual  display  from  the  driving  camera  scene  to  the  map  display. 
The  terrain,  targets,  and  hazards  were  counterbalanced  along  with  display  condition  to  control 
for  order  effects,  such  as  practice,  learning,  boredom,  or  fatigue.  The  display  options  were 
evaluated  for  effects  on  task  perfonnance,  workload,  and  usability,  based  on  objective 
performance  data,  data  collector  observations,  and  Soldier  questionnaires. 


4 


1.3  Objective 


The  objective  of  this  experiment  was  to  assess  the  effectiveness  of  a  toggle-based  3.5  in.  display 
and  the  multimodal  3.5  in.  display,  as  compared  to  a  6.5  in.  split  screen  display. 

Performance  measures  addressed  the  following: 

•  How  does  the  use  of  the  different  display  conditions  affect  the  number  of  driving  errors? 

•  How  does  the  use  of  the  different  display  conditions  affect  the  overall  course  completion 
times? 

•  How  does  the  use  of  the  different  display  conditions  affect  the  SA  of  the  operator  around 
the  vehicle? 


2.  Method 


2.1  Participants 

Thirty-three  Soldiers  from  the  Fort  Benning  OCS  volunteered  to  participate  in  the  study.  These 
Soldiers  had  experience  as  enlisted  Soldiers  and  came  from  varied  military  occupational 
specialties. 

2.2  Instruments  and  Apparatus 
2.2.1  TALON  Robot 

The  TALON  is  a  lightweight  robot  designed  for  missions  ranging  from  reconnaissance  to 
weapons  delivery  (figure  1).  Built  with  all-weather,  day/night,  and  amphibious  capabilities,  the 
TALON  can  operate  during  adverse  conditions  over  almost  any  terrain.  The  suitcase-portable 
robot  is  controlled  through  a  two-way  radio  frequency  line  from  a  portable  OCU  that  provides 
continuous  data  and  video  feedback  for  precise  vehicle  positioning.  It  was  developed  for  the 
Explosive  Ordnance  Disposal  Technology  Directorate  of  the  U.S.  Army’s  Armament  Research, 
Development,  and  Engineering  Center  at  Picatinny  Arsenal,  NJ,  by  the  engineering  and 
technology  development  firm  of  Foster-Miller.  The  TALON  began  being  used  in  military 
operations  in  Bosnia  in  2000.  It  was  deployed  to  Afghanistan  in  early  2002  and  has  been  in  Iraq 
since  the  war  started,  assisting  with  improvised  explosive  device  detection  and  removal. 
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Figure  1.  TALON  robot. 

For  this  experiment,  the  TALON  was  equipped  with  a  video  camera  that  enabled  Soldiers  to 
maneuver  the  vehicle  and  assess  enemy  activity  along  the  route  to  the  objective.  It  was  also 
equipped  with  a  global  positioning  system  (GPS)  that  informed  the  navigation  display  of  its 
position  and  orientation. 

2.2.2  Robotic  Vehicle  Displays 

DCS  Corporation  and  Tank  Automotive  Research  Development  and  Engineering  Center 
(TARDEC)  implemented  the  user  interface  and  protocols  for  all  the  aids  required  in  this 
experiment.  Three  different  display  conditions  were  used  to  conduct  this  experiment.  Each 
display  condition  was  plugged  into  the  existing  TALON  control  system  so  that  everything 
remained  constant  except  the  displays. 

Display  A  was  a  6.5-in.  display  that  provided  a  split  screen  capability.  The  top  portion  of  the 
screen  presented  the  driving  camera  information  and  the  bottom  portion  presented  the  map 
display  (figure  2).  Both  displays  could  be  viewed  simultaneously. 
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Figure  2.  Display  A,  split  screen. 

Display  B  was  a  3.5-in.  display  that  could  present  either  the  driving  camera  (figure  3)  or  the  map 
(figure  4).  Soldiers  toggled  between  the  displays  as  needed  by  pushing  a  single  toggle  button  on 
the  controls.  The  displays  had  to  be  viewed  sequentially. 


Figure  3.  Driving  camera  scene  for  Displays  B  and  C. 
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Figure  4.  Map  display  scene  for  Displays  B  and  C. 

Display  C  was  the  same  3.5-in.  display  used  for  Display  B  (figures  3  and  4)  with  the  addition  of 
a  tactile  belt  (see  paragraph  2.2.3  for  a  description  of  the  belt).  The  driving  camera  scene  was 
used  primarily,  but  the  Soldier  could  toggle  to  the  map  display  when  specific  location 
information  was  needed. 

2.2.3  Tactile  Belt 

The  TACTICS  tactile  system  was  developed  by  the  University  of  Central  Florida  (UCF)  under 
Defense  Advanced  Research  Project  Agency  contract  number  DAAE0703CL143  (figure  5).  It  is 
capable  of  remotely  conveying  covert  signals,  cues,  and  messages  by  touch.  The  system  for  this 
experiment  consisted  of  a  tactile  display  worn  around  the  waist  with  a  receiver  unit.  The  display 
itself  consisted  of  eight  tactile  drivers  that  created  a  strong  localized  sensation  on  the  body  and 
worked  similar  to  a  plunger.  The  eight  tactors  corresponded  to  the  eight  cardinal  directions.  The 
tactors  could  be  activated  individually,  sequentially,  or  in  groups;  and  the  duration  and  signal 
frequency  could  be  varied  to  provide  a  specific  sensation  or  to  create  unique  patterns  of 
vibration.  The  control  unit  received  wireless  signals  from  the  GPS  onboard  the  TALON  and 
converted  them  into  recognizable  patterns  of  vibration.  When  the  TALON  was  going  in  the 
correct  direction,  the  tactor  at  the  front  of  the  Soldier’s  waist  activated  (200  ms  on  and  1800  ms 
off).  Thus,  the  object  was  to  drive  the  robot  in  the  direction  that  kept  the  front  tactor  active. 
When  the  TALON  was  within  5  m  of  the  waypoint,  the  tactor’s  pulse  increased  to  100  ms  on, 
200  ms  off,  100  ms  on,  600  ms  off.  When  it  moved  to  within  2  m  of  the  waypoint,  the  waypoint 
was  considered  to  be  achieved  and  all  the  tactors  activated  for  3000  ms.  If  a  Soldier  went  off 
course  more  than  2  m,  the  tactor  that  corresponded  with  a  “steer  to”  direction  for  returning  to  the 
correct  course  route  activated  at  200  ms  on  and  1800  ms  off. 
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Figure  5.  TACTICS  tactile  belt  system. 


2.2.4  Tablet  Display 

The  visual  display  that  was  used  to  present  the  3.5  in.  map  and  camera  display  and  the  split 
screen  display  was  the  AMREL  Rocky  DR7-M  (figure  6).  It  is  9.8  in.  by  7.4  in.  and  weighs 

2.4  lb  (including  the  battery)  and  has  GPS  and  Bluetooth  integrated  options.  It  is  a  1024  by  768 
extended  graphics  array  (XGA). 


Figure  6.  Display. 
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2.2.5  Robotic  Navigation  Course 

The  training  portion  of  the  course  consisted  of  a  lane  that  allowed  the  Soldier  to  practice 
teleoperating  the  robot  around  a  complex  obstacle  in  each  direction  using  the  driving  camera  and 
control  unit.  It  also  consisted  of  another  section  with  waypoints  that  allowed  the  Soldier  to 
navigate  the  robot  to  the  waypoints  using  each  of  the  navigation  system  configurations.  The 
experiment  course  consisted  of  three  different  lanes.  This  allowed  the  Soldier  to  drive  on  a 
different  lane  with  each  display  condition.  The  total  length  of  each  lane  was  approximately 
100  m  (figure  7).  The  first  leg  of  each  lane  was  a  marked  path  approximately  30  m  long 
(figure  8).  A  complex  obstacle  was  placed  at  the  end  of  the  marked  path  that  required  the 
operator  to  navigate  around  it  (figure  9).  The  obstacle  did  not  show  up  on  the  map  display  so  the 
operator  was  required  to  drive  around  the  obstacle  and  continue  to  the  end  waypoint  (objective). 
The  second  leg  of  the  course  required  the  operator  to  drive  as  quickly  and  efficiently  as  possible 
using  the  GPS  feedback  to  the  final  waypoint.  Hand  signals  were  presented  by  a  data  collector 
who  was  moving  with  the  vehicle  to  monitor  driving  errors.  The  hand  signals  were  presented  up 
to  10  different  times  (depending  on  how  quickly  the  operator  navigated  the  course)  during  the 
final  leg  of  the  course.  The  data  collector  ensured  that  each  signal  was  presented  in  front  of  the 
driving  camera  so  that  it  could  be  clearly  seen  by  the  TALON  operator  if  he  was  attending  to  the 
driving  camera.  Soldiers  reported  when  they  saw  the  hand  signal  and  which  specific  signal  they 
saw  to  the  data  collector  sitting  next  to  them  in  the  tent,  who  was  also  timing  the  course 
completion.  Soldiers  teleoperated  the  TALON  from  inside  a  tent  during  training  and  actual 
navigation  course  negotiation,  which  prevented  them  from  teleoperating  the  vehicle  using  line  of 
sight  rather  than  the  driving  camera  display. 
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Figure  8.  Marked  path  from  first  leg  of  robotic  navigation  course. 


Figure  9.  Complex  obstacle. 


Figure  10.  Presentation  of  hand  signals. 
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2.2.6  Motion  Sickness  Questionnaire 

The  Motion  Sickness  Questionnaire  requires  the  Soldiers  to  describe  any  fonn  of  motion 
sickness  they  might  have  experienced  during  the  preceding  exercise  (Gianaros,  Muth,  Mordkoff, 
Levine,  and  Stern,  2001). 

2.2.7  Soldier  Subjective  Questionnaires 

Questionnaires  included  a  post-iteration  questionnaire  and  an  end  of  experiment  questionnaire. 
The  questionnaires  were  designed  to  elicit  Soldiers’  opinions  about  their  performance  and 
experiences  with  each  of  the  display  conditions.  The  post-iteration  questionnaires  had  the 
Soldiers  rate  the  display  concepts  and  training  on  a  7-point  semantic  differential  scale  ranging 
from  “extremely  good/easy”  to  “extremely  bad/difficult.”  The  end  of  experiment  questionnaire 
had  the  Soldiers  compare  and  contrast  the  display  concepts. 

2.2.8  NASA-TLX 

The  National  Aeronautics  and  Space  Administration  (NASA)-Task  Load  Index  (TLX)  is  a 
subjective  workload  assessment  tool,  which  allows  subjective  workload  assessments  on 
operator(s)  working  with  various  human-machine  systems  (Hart  and  Staveland,  1988).  It  uses  a 
multi-dimensional  rating  procedure  that  derives  an  overall  workload  score  based  on  a  weighted 
average  of  ratings  on  six  subscales.  These  subscales  include  mental  demands,  physical  demands, 
temporal  demands,  own  performance,  effort,  and  frustration.  It  can  be  used  to  assess  workload 
in  various  human-machine  environments  such  as  aircraft  cockpits;  command,  control,  and 
communication  workstations;  supervisory  and  process  control  environments;  simulations;  and 
laboratory  tests. 

2.2.9  SynWin 

The  SynWork  multitask  synthetic  work  program  (Elsmore,  1994)  is  a  battery  of  tasks  that 
represents  some  of  the  perceptual  and  cognitive  skills  required  in  many  kinds  of  complex  work. 

It  has  been  found  to  be  a  reliable  and  sensitive  measure  of  multitask  performance  in  a  variety  of 
settings  (Proctor,  Wang,  and  Pick,  1998).  SynWin  is  the  Windows-based  version  of  the  same 
program.  It  includes  four  tasks  that  can  be  set  up  to  be  presented  simultaneously.  The  tasks 
include  a  simple  memory  task,  an  arithmetic  computation  task,  a  visual  monitoring  task,  and  an 
auditory  monitoring  task.  The  memory  task  briefly  shows  a  string  of  letters  and  the  participant 
has  several  seconds  to  memorize  the  string  before  it  disappears.  After  this,  a  single  letter  appears 
and  the  participant  must  indicate  if  that  letter  is  part  of  the  original  string.  The  arithmetic 
computation  task  displays  two  or  three  numbers  (three  digits  long)  and  the  participant  must  add 
the  numbers.  The  visual  monitoring  task  presents  a  fuel  gauge  and  a  pointer  ticks  down  from 
100  to  0.  The  participant  attempts  to  click  on  the  gauge  to  “refill”  it,  causing  it  to  jump  back  to 
100.  The  closer  the  pointer  is  to  0  when  the  gauge  is  clicked,  the  greater  the  number  of  points 
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awarded.  However,  if  the  participant  waits  too  long,  there  is  a  large  score  penalty.  The  audio 
monitoring  task  plays  one  of  two  tones  at  set  intervals  and  the  participant  must  listen  for  a 
particular  tone.  When  that  tone  is  played,  the  participant  clicks  an  alert  button.  When  the  tasks 
are  presented  simultaneously,  each  is  presented  in  a  separate  quadrant  on  the  screen  (figure  11). 


SynWin  [1  0  03]:  UNKNOWN,  Session  2 


List  Box  --> 


Click  the  List  Box  to  view  list 


A  P  J  Z  I  U 


Probe  Letter  ~> 


Click  YES  if  Probe  letter  is  in  the  list,  NO  if  it  isn't 


Click  to  refuel 


Digit 
Adjustment 
controls  --> 


715 
+  551 

1266 


+1 

+ 

1+ 

+1 

-J 

-1 

1- 

1- 

Adjust  digits  until 
answer  is  correct 
then  click  DONE 


DON 


A 


■ 

ALERT 


Click  the  ALERT  button  when 
you  hear  the  HIGH  tone 


Figure  11.  SynWin  tasks  presented  in  the  four  quadrants. 


2.3  Procedures 

2.3.1  Demographics 

Demographic  data,  as  well  as  data  concerning  the  Soldiers’  Anny  and  robotic  experience,  were 
documented  for  each  Soldier. 

2.3.2  Training 

The  experiment  Soldiers  reported  in  groups  of  six  for  one  day  each.  Before  the  first  training 
presentation,  experiment  Soldiers  received  a  roster  number,  which  was  used  to  identify  them 
throughout  the  evaluation.  They  were  also  given  an  operations  order  that  explained  the  robotic 
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mission  that  they  would  undertake  during  the  experiment.  Driving  requirements,  obstacle 
negotiation  requirements,  and  identification  of  hand  signals  were  all  explained  and  any  questions 
the  Soldiers  had  concerning  the  experiment  were  answered.  A  representative  from  Foster-Miller 
trained  the  Soldiers  in  the  use  of  the  TALON  robot.  Soldiers  practiced  teleoperating  and 
navigating  the  robot  on  a  training  course  that  was  similar  to  the  actual  course  used  during  the 
experiment.  This  course  required  driving,  obstacle  negotiation,  and  navigation.  Soldiers  were 
trained  using  each  of  the  displays.  Before  they  could  begin  the  actual  experiment  trials,  they  had 
to  demonstrate  proficiency  with  the  system  and  displays. 

2.3.3  Teleoperation  Trials 

Soldiers  were  assigned  to  displays  and  lanes  according  to  the  matrix  presented  in  table  1 .  The 
matrix  was  constructed  using  a  Williams  Squares  design  for  treatment  conditions,  lanes,  and 
preceding  device  (so  that  no  device  always  followed  another)  (Williams,  1949). 

Table  1.  Order  of  treatments  and  lanes. 


Roster 

Iteration 

1 

2 

3 

Display 

Course 

Display 

Course 

Display 

Course 

1,7,  13,  19 

C 

3 

B 

2 

A 

1 

2,  8,  14,  20 

A 

2 

C 

1 

B 

3 

3,  9,  15,21 

B 

1 

A 

3 

C 

2 

4,  10,  16,  22 

A 

1 

B 

2 

C 

3 

5,  11,  17,  23 

B 

3 

C 

1 

A 

2 

6,  12,  18,  24 

C 

2 

A 

3 

B 

1 

The  Soldiers  were  required  to  teleoperate  the  robot  within  the  marked  leg  of  the  course  as 
quickly  and  efficiently  as  possible,  negotiate  around  the  obstacle  in  the  path,  and  navigate  to  the 
waypoint.  A  15-rnin  time  limit  was  given  for  completion  of  each  lane.  A  data  collector  walked 
behind  the  TALON  on  each  trial,  sending  hand  signals  and  documenting  the  number  of  driving 
errors  (backups,  turning  in  the  wrong  direction,  etc.)  on  each  lane.  A  data  collector  sitting  next 
to  the  operator  documented  the  number  of  hand  signals  correctly  identified  and  the  overall  course 
completion  time. 

Upon  completion  of  the  navigation  course  with  each  navigation  system,  the  Soldiers  were  given 
a  questionnaire  designed  to  assess  their  perception  of  the  training  adequacy  and  their  ease  of 
performance  with  the  navigation  device.  Questions  about  the  amount  of  practice  time  given,  the 
level  of  detail  presented,  the  adequacy  of  training  aids,  the  ease  of  operation,  and  their  SA  were 
asked.  The  motion  sickness  questionnaire  and  the  NASA-TLX  were  also  given  after  each  course 
iteration.  At  the  end  of  the  day,  Soldiers  completed  a  questionnaire  comparing  the  three  different 
navigation  concepts. 
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2.3.4  SynWin 


The  SynWin  was  administered  to  each  Soldier  to  assess  their  ability  to  multitask.  Soldiers  were 
trained  first  with  a  PowerPoint  presentation,  then  with  a  5  min  practice  session  where  they 
started  first  with  the  memory  task.  After  1  min,  the  math  task  was  added.  After  another  minute 
the  fuel  gauge  task  was  added,  then  after  another  minute,  the  audio  task  was  added.  Soldiers  had 
no  problem  comprehending  the  nature  of  the  individual  tasks  and  the  need  to  manage  the  tasks  in 
such  a  way  as  to  maximize  the  composite  score.  After  training,  the  Soldiers  completed  the 
SynWin  trial  and  scores  for  each  of  the  tasks  and  a  composite  score  were  documented. 


3.  Results 


3.1  Demographics 

The  original  sample  size  of  Soldiers  was  33.  This  sample  was  reduced  to  25  because  3  of  the 
Soldiers  failed  to  meet  the  proficiency  requirements  during  the  time  limit  allowed  for  the  training 
course,  and  the  other  5  experienced  a  robotic  malfunction  or  loss  of  GPS  signal  on  the  navigation 
course  during  one  or  more  of  their  iterations  that  adversely  affected  their  completion  times. 

Since  this  was  a  within  subject  evaluation,  their  times  from  the  other  trials  also  had  to  be  deleted. 
The  25  OCS  Soldiers  who  participated  in  this  experiment  averaged  4  years  in  the  military.  They 
came  from  a  variety  of  Anny  fields,  including  Infantry,  Aviation,  Armor,  Administration, 

Vehicle  Maintenance,  Transportation,  and  Logistics.  Six  of  the  Soldiers  had  been  deployed  in  a 
combat  area.  Fourteen  of  the  Soldiers  had  used  a  GPS  and  the  majority  classified  themselves  as 
having  intermediate  land  navigation  experience.  While  only  9  had  robotic  experience,  all  but  2 
Soldiers  had  video  game  experience.  The  weights  of  the  Soldiers  ranged  from  the  40th  percentile 
female  to  the  98th  percentile  male  (130  to  230  lb).  Their  heights  ranged  from  the  20th  percentile 
female  to  the  99th  percentile  male  (62  to  75  in.).  The  average  age  of  these  Soldiers  was  30  years. 
Seventeen  Soldiers  were  left-eye  dominant  and  12  wore  prescription  lenses.  See  the  appendix 
for  detailed  demographic  information. 

3.2  Training  Evaluation 

The  participants  were  generally  positive  about  the  quality  of  training  on  the  use  of  the  controls 
and  displays.  There  were  several  comments  about  activities  that  were  challenging  to  leam. 

Some  of  the  Soldiers  mentioned  the  difficulty  in  coping  with  the  time  lag  in  the  GPS  signal. 

They  had  some  difficulty  in  judging  distances  using  the  display  screen.  Several  participants 
noted  that  it  took  some  time  to  get  the  feel  of  the  joystick  control. 

Detailed  responses  to  questions  pertaining  to  the  training  are  included  in  the  post  iteration 
questionnaire  in  the  appendix. 
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3.3  Performance  Data 


3.3.1  Course  Times 

Table  2  shows  the  mean  course  time  for  the  three  displays. 


Table  2.  Mean  course  completion  times. 


Display 

min:sec 

Mean 

SD 

A 

6:06 

2:33 

B 

7:29 

2:35 

C 

6:26 

2:37 

Figure  12,  which  shows  the  distribution  of  course  times  summed  across  all  three  display 
conditions,  demonstrates  that  the  course  times  were  highly  skewed.  The  Pearson  coefficient  of 
skew  for  course  times  was  Skp  =  0.68.  Winer,  Brown,  and  Michels  (1991)  recommend  log 
transformations  to  address  positive  skewness  that  frequently  occurs  when  the  dependent  variable 
is  time  required  to  complete  a  task.  It  is  important  to  re-examine  the  distribution  after 
transformation  to  ensure  that  the  skewness  is  substantially  reduced  (Tabachnick  and  Fidell, 
2007).  A  log  10  transfonnation  (Winer,  Brown,  and  Michels,  1991)  was  used  to  produce  a 
distribution  that  more  closely  approximates  the  normal  distribution,  thus  better  satisfying  the 
assumptions  of  analysis  of  variance  (ANOVA).  As  shown  in  figure  13,  the  loglO  transformation 
was  successful  in  providing  a  better  approximation  to  the  normal  curve.  The  Pearson  coefficient 
of  skew  for  the  log  10  course  times  was  Skp  =  0.17. 
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Figure  12.  Course  completion  times  (sec),  all  display  conditions. 


2.39  2.49  2.59  2.69  2.79  2.89 


Figure  13.  LoglO  course  completion  times  (sec),  all  display  conditions. 
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Table  3  shows  the  log  10  means. 


Table  3.  Log  10  mean  course  completion  times. 


LoglO 

Display 

Mean 

SD 

A 

2.5340 

0.1565 

B 

2.6284 

0.1476 

C 

2.5526 

0.1757 

A  repeated  measures  ANOVA  conducted  on  the  log  10  course  times  indicated  that  there  was  a 
significant  difference  among  the  means:  F(2,48)  =  4.34, p  =  .018,  rfp  =  .153. 

Since  we  were  investigating  whether  the  smaller  screen  size  can  be  used  as  effectively  as  the 
larger  one,  the  performance  times  with  each  of  the  3.5  in.  screen  versions  was  compared  to  the 
time  with  the  6.5  in.  screen.  For  such  a  priori  comparisons,  t-tests  are  conducted  using  the  error 
tenn  from  the  ANOVA  to  calculate  a  pooled  error  term  for  the  t-tests  (Winer,  Brown  and 
Michels,  1991).  The  results  of  the  planned  comparisons  are  shown  in  table  4. 


Table  4.  Planned  comparisons,  course  completion  times. 


Pair 

t 

df 

logA  -  logB 

3.93a 

24 

logA  -  logC 

0.78 

24 

logB  -  logC 

3.16a 

24 

a p  <  .01,  1-tailed 


Hypothesis  HO  1  that  said  Soldiers  will  perfonn  the  robot  navigation  task  more  quickly  with  the 
split  screen  (A)  than  the  toggle  screen  (B)  is  supported.  The  first  portion  of  hypothesis  H04  that 
stated  Soldiers  will  perform  the  robot  navigation  task  more  quickly  with  the  multimodal  display 
(C)  than  with  the  toggle  display  (B)  is  also  supported.  The  portion  of  hypothesis  H04  that  stated 
there  will  be  no  difference  between  the  multimodal  and  the  split  screen  displays  is  supported  as 
well. 

3.3.2  Situation  Awareness 

Table  5  shows  the  mean  percentage  of  hand  signals  correctly  identified  with  each  display  type. 
(The  data  are  presented  as  percentages  because  the  number  of  hand  signals  presented  varied  from 
5  to  10.)  The  omnibus  differences  among  the  means  approached  statistical  significance: 

F(2,48)  =  2.96,  p  =  .061,  q"p  =0.061.  The  pre-planned  comparisons  can  be  found  in  table  6. 
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Table  5.  Mean  percentage  of  hand  signal  identified. 


Display 

Mean 

SD 

A 

96.8 

8.5 

B 

90.0 

13.2 

C 

93.2 

11.4 

Table  6.  Planned  comparisons,  percentage  of  hand  signal  identified. 


Pair 

t 

df 

A-B 

3.44a 

24 

A-C 

1.82 

24 

B-C 

1.62 

24 

a p  <  .01,  1-tailed 


Hypothesis  H03  that  said  Soldiers  will  identify  hand  signals  more  quickly  and  accurately  with 
the  split  screen  (A)  than  with  the  toggle  screen  (B)  is  supported.  The  first  portion  of  hypothesis 
H06  that  stated  Soldiers  will  identify  hand  signals  more  quickly  and  accurately  with  the 
multimodal  display  than  with  the  toggle  display  is  not  supported.  The  portion  of  hypothesis  H03 
that  stated  there  will  be  no  difference  between  the  multimodal  and  the  split  screen  displays  is 
supported. 


3.3.3  Driving  Errors 

The  mean  number  of  driving  errors  with  each  display  type  is  shown  in  table  7.  In  each  condition, 
participants  averaged  less  than  one  error  per  course  completion.  The  difference  among  the 
means  was  not  statistically  significant:  F(2,48)  <  1.00 ,p  =  .748,  rfp  =  .012. 


Table  7.  Mean  number  of  driving  errors. 


Display 

Mean 

SD 

A 

0.16 

0.47 

B 

0.12 

0.33 

C 

0.08 

0.28 

Hypothesis  H02  that  said  Soldiers  will  perfonn  the  robot  navigation  task  with  fewer  errors  with 
the  split  screen  than  with  the  toggle  screen  is  not  supported.  Neither  is  H05,  which  stated  that 
Soldiers  will  perform  the  robot  navigation  task  with  fewer  errors  with  the  multimodal  display 
than  with  the  toggle  display.  There  was  no  difference  between  the  multimodal  and  the  split 
screen  displays. 


3.3.4  SynWin 

Descriptive  statistics  for  the  SynWin  task  are  shown  in  table  8.  The  SynWin  scores  did  not 
correlate  significantly  with  course  completion  times. 
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Table  8.  SynWin  subtest  and  composite  means. 


Subtest 

Mean 

SD 

Memory 

110.4 

125.3 

Math 

70.4 

60.0 

Visual 

114.0 

28.5 

Alert/audio 

52.8 

41.8 

Composite 

347.6 

182.1 

3.3.5  NASA-TLX 

The  results  of  NASA-TLX  workload  questionnaire  are  summarized  in  figure  14.  The  graph 
clearly  demonstrates  that  there  was  very  little  difference  among  display  conditions  for  any  of  the 
workload  categories,  including  total  workload. 


Figure  14.  NASA-TLX  means  by  display  condition. 


3.4  Subjective  Results 

3.4.1  Post-iteration  Questionnaire 

Task  difficulty  ratings  for  the  navigation  course  were  generally  worse  for  the  toggle  display  than 
for  the  split  screen  or  tactile  displays.  The  Soldiers  were  very  positive  about  the  wealth  of 
information  provided  by  the  split  screen  display.  They  were  also  very  positive  about  the  tactile 
information  system.  Several  commented  that  they  liked  being  able  to  give  their  full  attention  to 
the  driving  camera  display  and  rely  on  the  tactile  belt  for  the  navigation  information.  One 
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Soldier  commented  that  the  tactile  display  allowed  him  to  concentrate  on  objects  closer  to  the 
robot  because  he  did  not  have  to  use  distant  objects  to  orient  for  direction  of  travel.  The  Soldiers 
were  also  generally  positive  about  using  the  toggle  display,  but  some  were  annoyed  by  the  need 
to  toggle  between  views. 

The  participants  rated  their  SA  as  being  lower  with  the  toggle  display  than  with  the  split  screen 
or  tactile  displays.  They  complained  that  toggling  back  and  forth  between  views  caused  them  to 
miss  some  of  the  hand  signals.  Some  were  more  efficient  than  others  in  glancing  at  the  map 
display  and  quickly  toggling  back  to  the  camera  display. 

Some  of  the  problems  experienced  by  the  Soldiers  were  related  to  the  camera  on  the  vehicle  and 
not  the  specific  display  type.  For  example,  there  were  complaints  about  the  quality  of  the  video 
display  and  several  Soldiers  commented  negatively  about  the  blind  spots  around  the  vehicle 
(because  there  was  a  single  camera).  Also  a  few  Soldiers  expressed  problems  with  the  camera’s 
fixed  position,  which  caused  them  to  be  unable  to  see  the  road  when  negotiating  an  upward  slope 
(when  the  vehicle  is  on  a  forward  slope,  the  camera  points  up  because  it  is  in  a  fixed  position). 

Soldiers  experienced  difficulty  with  the  latency  between  the  vehicle  and  driving  camera  display 
and  between  the  vehicle  position  and  GPS  updates.  Most  of  the  Soldiers  learned  to  compensate 
somewhat  for  the  latency  but  several  did  not.  These  Soldiers  consistently  overcompensated, 
which  resulted  in  increased  course  times  and  longer  routes  to  the  waypoints. 

The  egocentric  GPS  caused  a  problem  for  one  Soldier.  He  stated  that  during  the  time  he  toggled 
to  the  map  display  to  get  reoriented  after  negotiating  around  an  obstacle,  the  map  rotated  without 
him  realizing  it.  Thus,  when  he  toggled  back  to  the  map  display,  he  started  off  in  the  wrong 
direction. 

Soldiers’  overall  mean  ratings  of  the  split  screen  and  the  tactile  system  were  very  good.  Their 
overall  mean  ratings  of  the  toggle  system  were  neutral.  The  majority  of  the  Soldiers  felt  that  the 
split  screen  and  tactile  displays  were  both  very  good  for  robotic  navigation. 

3.4.2  End  of  Experiment  Questionnaire 

The  participants  were  asked  their  preference  for  each  pair  of  displays  (i.e.,  Display  A  vs.  Display 
B,  Display  A  vs.  Display  C,  and  Display  B  vs.  Display  C).  Figure  15  shows  the  proportion  of 
times  that  each  display  type  was  selected  over  one  of  the  other  display  types.  This  type  of 
scaling,  referred  to  as  the  Thurstone  method  of  paired  comparisons,  is  designed  to  yield  an  equal- 
interval  scale  of  preference.  The  split  screen  was  the  most  preferred  display,  closely  followed  by 
the  multimodal,  and  the  toggle  was  the  least  preferred.  Several  participants  suggested  that  the 
optimal  display  would  combine  the  split  screen  with  the  tactile  belt. 
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Figure  15.  Paired  display  preferences. 

The  Soldiers  had  a  number  of  suggestions  for  enhancing  the  quality  of  the  displays.  They  asked 
for  better  resolution  and  brightness,  and  less  jitter  in  the  video  picture.  There  was  a  general 
consensus  that  robotic  driving  would  be  easier  if  the  latencies  were  reduced  or  eliminated  (both 
the  latency  in  the  GPS  refresh  rate  and  the  latency  between  the  driving  inputs  and  the  displayed 
movements).  Two  Soldiers  stated  that  the  tactile  signals  were  not  strong  enough  and  one  stated 
that  the  strength  of  the  signals  was  irritating 

3.5  Engineering  Lessons  Learned 

Accurate  GPS  positioning  of  the  robot  was  key  to  this  experiment.  Variations  of  the  GPS  drift 
per  day  and  during  the  day  greatly  affected  the  ability  of  the  operators  to  successfully  accomplish 
their  task  because  of  the  relatively  short  distances  used  in  this  experiment.  The  operator’s  only 
indication  of  the  target  position  was  the  waypoint  depiction  on  the  map  relative  to  the  robot 
depiction  on  the  map.  The  waypoint  position  was  always  a  fixed  latitude/longitude  position 
while  the  robot  position  was  a  GPS  returned  latitude/longitude.  The  GPS  position  for  the  same 
physical  position  tended  to  drift  during  the  day  by  approximately  ±15  m.  The  drift  made  the 
physical  marking  for  the  waypoints  differ  from  the  nominated  latitude/longitude  of  the  map 
depiction  of  the  waypoint.  Because  of  this  GPS  error,  operators  tended  to  drive  slightly  different 
courses  to  achieve  the  waypoints,  sometimes  forcing  a  data  collector  to  call  that  the  waypoint 
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was  achieved  when  the  operator  drove  past  it  to  the  left  or  right.  The  GPS  also  has  a  tendency  to 
“pop”  or  jump  by  ±5  m,  which  confused  the  operator  concerning  the  position  of  the  robot 
relative  to  the  target  waypoint.  Coupled  with  the  errors  inherent  in  GPS  were  various  technical 
problems  associated  with  the  GPS  system  mounted  on  the  TALON  for  this  experiment,  which 
was  inexpensive  and  tended  to  be  inaccurate. 

Accurate  robotic  compass  headings  were  necessary  to  reduce  operator  confusion  when  reading 
the  robot’s  position  on  the  map.  Initially,  problems  were  experienced  with  the  heading  reported 
by  the  robots,  which  made  the  use  of  the  robot  depiction  on  the  map  nonsensical.  Correct 
calibration  of  the  robot’s  onboard  compass  solved  this  issue. 

The  tactile  direction  signals  were  calculated  from  the  GPS  position  reported  by  the  robot  relative 
to  the  target  waypoint  position.  The  update  rate  of  the  GPS  was  approximately  once  per  second. 
Thus,  there  is  a  relationship  between  the  rate  of  tactile  signals  and  the  speed  with  which  the  robot 
changes  its  position  and  that  position  change  is  reported  by  the  GPS.  The  signals  used  for  our 
experiment  need  to  be  modified  for  future  experiments  to  allow  for  the  timing  between  the  GPS 
updates  to  reduce  operator  confusion. 

The  robots  used  required  manual  calibration  of  the  internal  compasses.  Various  Foster-Miller 
robot  software  patches  were  required  to  decode  the  correct  GPS  message  packets.  The  Foster- 
Miller  GPS  cabling  had  to  be  rerouted  away  from  the  drive  motors  to  reduce  interference  that 
precluded  the  GPS  position  from  updating  until  the  drive  motors  were  stopped. 

Inexperienced  operators  tended  to  neutral  steer,  changing  the  robot  direction  on  the  spot.  This 
stopped  one  side  of  the  robot  from  moving  by  actively  holding  the  motor  in  its  current  position 
while  the  drive  motor  on  the  opposite  side  turned  the  robot  around.  When  this  happened 
frequently,  the  motors  overheated.  When  the  drive  motors  overheated,  they  shutdown  until  they 
cooled  enough  to  operate  again.  This  gave  the  impression  that  the  operator  no  longer  had 
control. 


4.  Discussion 


The  results  from  this  experiment  demonstrate  the  efficacy  of  the  addition  of  a  tactile  belt  to  a 
robotic  OCU  for  dismounted  operations.  Soldiers  performed  equally  as  well  with  the  smaller 
display  size  that  included  a  tactile  navigation  display  (the  multimodal  display)  as  they  did  with 
the  larger  split  screen  display.  Sequential  operation  of  a  visual  driving  camera  display  and  a 
visual  map  display  was  not  as  effective.  Specifically,  Soldiers’  timed  performance  with  the  split 
screen  and  the  multimodal  displays  was  significantly  better  than  their  timed  perfonnance  with 
the  toggle  display.  There  was  no  difference  between  their  timed  performance  with  the  split 
screen  and  multimodal  displays.  While  there  was  no  difference  in  the  Soldiers’  driving  errors 
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with  any  of  the  displays,  it  should  be  noted  that  there  were  very  few  driving  errors  committed  on 
this  short  course.  More  errors  might  be  present  if  a  longer  course  with  more  obstacles  and  terrain 
features  was  used  and  differences  between  displays  might  become  more  apparent. 

As  hypothesized,  the  Soldiers  identified  more  hand  signals  with  the  spilt  screen  display  than  with 
the  toggle  display  and  there  was  no  difference  between  the  number  of  hand  signals  identified 
with  the  split  screen  and  multimodal  displays.  While  no  differences  were  found  between  the 
toggle  and  multimodal  displays  in  terms  of  the  operators’  SA,  further  study  is  warranted.  The 
availability  of  the  robots  limited  the  scope  of  this  experiment  in  terms  of  SA  trials.  The  limited 
number  of  hand  signals  given  in  this  experiment  and  the  ease  of  detection  of  the  hand  signals 
resulted  in  somewhat  of  a  ceiling  effect.  The  original  plan  included  a  leg  of  the  course  dedicated 
to  SA  trials  that  necessitated  the  operators  to  search  for  infonnation  along  the  route.  The  search 
was  designed  to  require  the  operators  to  look  carefully  along  both  sides  of  the  route  and  to  look 
for  objects  that  had  low  contrast  with  the  surroundings.  This  task  would  have  required  more 
time  in  the  searching  mode  and  would  have  allowed  less  time  for  looking  at  a  visual  map.  The 
more  demanding  SA  task  also  would  not  have  provided  an  attention-getting  movement  cue  (like 
when  the  data  collector’s  hands  moved  in  front  of  the  camera).  Thus,  if  the  operator  workload 
had  been  higher,  with  more  competition  for  visual  resources,  results  may  have  favored  the 
multimodal  option  to  a  greater  degree.  Since  much  of  the  navigation  task  depends  on  ambient 
and  focal  visual  information  processing,  Wicken’s  MRT  predicts  that  workload  for  this  task  can 
be  offloaded  to  a  different  sensory  channel  (Boles,  2001;  Wickens,  2002).  This  would  free  the 
visual  channel  for  driving  and  searching,  thus  theoretically  resulting  in  higher  SA.  Wickens 
(2002)  notes  that  when  tasks  are  complementary,  there  will  be  less  competition  for  resources  and 
more  efficient  time  sharing. 

Egocentric  map  perspectives  like  the  one  used  in  this  experiment  have  been  found  to  be  more 
efficient  in  route-guiding  situations  than  exocentric  maps  (Porathe,  2007;  Hermann,  Bieber,  and 
Duesterhoeft,  2003).  However,  this  type  of  map  was  problematic  during  this  experiment  when 
operators  had  to  attend  to  a  driving  camera  display  and  did  not  have  the  opportunity  to  pay 
constant  attention  to  the  map  display.  Seager  and  Stanton-Fraser  (2007)  found  this  to  be  true 
when  their  experiment  demonstrated  that  users  find  it  difficult  to  recognize  a  map  that  rotates 
automatically  when  they  are  not  looking  at  the  map.  The  addition  of  a  tactile  navigation  display 
would  alleviate  this  problem. 

Two  Soldiers  in  this  experiment  stated  that  the  tactile  signals  were  not  strong  enough  and  one 
stated  that  the  strength  of  the  signals  was  irritating.  Previous  research  with  tactile  displays 
indicates  that  tactor  intensity  strength  should  be  adjustable  to  account  for  individual  differences, 
stress,  and  differences  task  intensity  levels  (Redden,  Carstens,  Turner,  and  Elliott,  2006;  Redden 
and  Elliott,  2007;  van  Erp,  2007,  2002). 
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5.  Recommendations 


Further  experimentation  using  a  tactile  belt  for  dismounted  robotic  navigation  is  warranted. 
While  this  experiment  demonstrated  that  adding  a  tactile  belt  to  the  robotic  display  allowed  the 
use  of  a  smaller  screen  size,  more  experimentation  that  addresses  the  SA  of  the  operator  is 
needed  because  MRT  suggests  that  a  tactile  display  might  provide  increased  SA  around  the 
vehicle  over  a  split  screen  display. 

A  possible  solution  for  the  robotic  GPS  problems  would  be  to  include  an  inertial  positioning 
system  in  addition  to  the  GPS  on  the  robot  to  compensate  for  dropped  and  popped  GPS 
positions.  An  inertial  system  is  used  on  larger  robotic  vehicles  for  exactly  these  reasons.  Use  of 
a  GPS  only  system  is  useful  for  single  estimations  of  position  but  not  so  useful  in  a  continuous 
accurate  positioning  system. 

The  tactile  signals  used  for  robotic  navigation  need  to  be  modified  to  allow  for  the  timing 
between  GPS  updates  because  of  the  relationship  between  the  rate  of  the  tactile  signals  sent  by 
the  belt  and  the  speed  with  which  the  robot  changes  its  position  and  that  position  change  is 
reported  by  the  GPS.  The  tactile  belt  should  be  modified  so  that  individual  Soldiers  have  the 
capability  of  increasing  or  decreasing  the  tactor  strength  based  upon  their  own  pain  and 
discrimination  thresholds  or  upon  the  intensity  of  the  situation. 
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Appendix.  Soldier  Questionnaire  Results 


DEMOGRAPHICS 
SAMPLE  SIZE  =  25 


MOS: 

09S  -9 

42A  -  3  94R  -  1 

1  IB  -2 

88  -  1  97E  -1 

19D  -  1 

92A  -  1  OCS  -  2 

25V  -  1 

92Y  -  1  NR  -  1 

35P  -  1 

RANK: 

DUTY  POSITION 

E-4  -5 

Avionic  Systems  Maintainer 

1 

E-5  -  13 

Combat  Camera 

1 

E-6  -5 

Motor  Pool 

1 

E-7  -  1 

OCS 

13 

NR  -  1 

Human  Resources  NCO 

2 

Sniper  Section  NCO 

1 

Supply 

2 

No  response 

4 

1 .  What  is  your  age?  30  years  (mean  )  -  22-38  range 

2.  What  is  your  height?  69  inches  (mean)  -  62-75  range 

3.  What  is  your  weight?  180  lbs  (mean)  -  130-230  range 

4.  With  which  hand  do  you  most  often  write?  Right  -  2 1  Left  -  3  NR  -  1 

5.  With  which  hand  do  you  most  often  fire  a  weapon?  Right -20  Left -5 

6.  Do  you  wear  prescription  lenses?  Yes  -  12  No  -  13 


7.  If  Yes,  which  do  you  most  often  wear?  Glasses  -  6 

Contacts  -  6 

8.  Which  is  your  dominant  eye? 

Right  -  17 

Left  -  7  NR 

9.  Have  you  ever  used  a  GPS  system? 

Yes  -  14 

No -10  NR 

10.  If  Yes,  what  kind  of  GPS? 

Automotive  Nav  System 

3 

Garmin 

9 

Cell  phone 

1 

PLGR 

1 

No  response 

10 

29 


1 1 .  How  many  months  in  military? 

48  (mean) 

12.  Have  you  had  an  infantry-related  job? 

No -21 

Yes -3 

NR  -  1 

13.  How  many  months  in  infantry-related  job? 

93  (mean) 

14.  Have  you  been  a  fire  team  leader? 

No -22 

Yes -3 

15.  How  many  months  as  a  fire-team  leader? 

22  (mean) 

16.  Have  you  been  a  squad  leader? 

No  -  15 

Yes  -  10 

17.  How  many  months  as  a  squad  leader? 

20  (mean) 

18.  Have  you  been  deployed  overseas? 

No  -  14 

Yes  -  10 

NR  -  1 

19.  How  many  months  deployed  overseas? 

20  (mean) 

20.  Have  you  been  deployed  in  a  combat  area? 

No  -  19 

Yes  -  6 

2 1 .  How  many  months  deployed  in  combat  zone? 

13  (mean) 

None 

Beginner  Intermediate 

Expert  NR 

(number  of  responses) 

Land  navigation 

0 

8 

12 

5  0 

Operating  ground  unmanned  vehicles 

13 

9 

1 

1  1 

Operating  unmanned  aerial  vehicles 

23 

1 

0 

0  1 

Target  detection  and  identification 

8 

10 

3 

3  1 

Playing  commercial  video  games 

2 

6 

9 

8  0 

Training  with  Army  video  simulations 

6 

11 

5 

2  1 
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POST  ITERATION  QUESTIONNAIRE 


SAMPLE  SIZE  =  25 

A  (Spilt  Screen);  B  (Toggle);  C  (Tactile); 

1.  Using  the  scale  below,  please  rate  your  ability  to  perform  each  of  the  following 
navigation  tasks  based  on  your  experience  with  the  display  that  you  just  used: 


1 

2 

3 

4 

5 

6 

7 

Extremely  difficult 

Very  difficult 

Difficult 

Neutral 

Easy 

Very  easy 

Extremely  easy 

NAVIGATION  TASKS 

MEAN  RESP< 

ONSE 

A 

B 

C 

a.  Move  the  robot  in  the  correct  direction 

4.96 

4.72 

5.37 

b.  Identify  any  other  terrain  features  that  might  have  an  adverse 
effect  on  the  ability  of  the  robot  to  maneuver  through  the  terrain 

4.65 

4.80 

5.09 

c.  Locate  the  robotic  vehicle  on  the  map 

6.05 

6.00 

5.96 

d.  Determine  the  robot's  orientation  on  the  map 

6.00 

5.74 

5.91 

e.  Determine  when  you  need  to  make  a  course  correction 

5.55 

4.83 

6.00 

f.  Identify  if  you  are  on  the  correct  course 

5.55 

4.75 

6.18 

g.  Navigate  far  enough  ahead  to  plan  route  in  advance 

5.58 

4.80 

5.67 

h.  Navigate  well  enough  to  drive  at  slowest  speeds 

5.95 

5.39 

6.05 

i.  Navigate  well  enough  to  drive  at  medium  speeds 

5.46 

5.04 

5.54 

j.  Navigate  well  enough  to  drive  at  fastest  speeds 

4.92 

4.25 

4.58 

k.  Navigate  and  drive  simultaneously 

5.48 

4.20 

5.43 

1.  Navigate  around  obstacles 

5.17 

4.88 

5.33 

m.  Return  to  the  correct  route  after  navigating  around  obstacles 

5.58 

5.00 

5.83 

n.  Stay  on  course 

5.52 

4.64 

6.00 

o.  Overall  ability  to  perform  driving  and  navigation  tasks 

5.59 

4.67 

5.87 

Comments  No.  of  Responses 

A  (Split  Screen) 

Good  system,  course  corrections  are  made  easily  and  quickly.  1 

Being  able  to  see  the  map  and  the  camera  feed  at  the  same  time  was  very  helpful  1 

with  efficiency  and  not  missing  any  vital  information. 

I  found  that  the  ability  to  navigate  was  greatly  improved  with  the  split  screen  rather  1 
than  the  toggle  screen. 

Helpful  to  have  both  screens  in  order  to  focus  on  objective  and  not  to  miss  anything  1 
on  the  video  feed. 

The  spilt  display  helps  to  determine  which  course  to  take.  1 
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Comments 


No.  of  Responses 


This  was  the  easiest  setup  to  utilize.  The  ability  to  have  everything  on  one  screen,  1 
with  no  toggling  was  a  major  impact  on  the  overall  effectiveness  of  the  mission. 

The  tasks  were  fairly  comfortable  to  perfonn.  I  can't  say  it  was  easy  because  I  had  to  1 
make  adjustments  and  corrections.  I  can't  say  it  was  difficult  because  the 
adjustments  made  were  not  hard  to  do.  Overall,  it  was  a  neutral  process  that  was 
not  easy  nor  was  it  difficult. 

Controls  are  squishy,  but  the  split  screen  makes  driving  and  navigation  easier.  1 

The  time  lag  from  controller  to  robot  was  the  hardest  part  about  navigating  the  robot  1 
during  the  task.  The  moisture  on  the  lens  was  not  that  bad  for  the  task. 

I  had  little  trouble  orienting  myself  even  when  thrown  off  course  by  the  obstacle.  1 

Only  when  I  came  within  20  meters  of  the  objective  did  I  become  unsure  of  its 
location  as  it  was  behind  the  tent  which  I  felt  slightly  uncomfortable  navigating 
around.  In  reality,  I  was  right  on  line  the  entire  time. 

Battery  on  robot  ran  low  while  perfonning  test  making  maneuvering  more  difficult.  1 

Black  and  white  screen  made  it  hard  to  identify  obstacles  and  terrain  features.  1 

Bushes  will  roll  the  robot  over.  1 

B  (Toggle) 

It  was  fair.  1 

The  main  benefit  was  that  you  had  a  decent  enough  camera  angle  to  see  far  enough  1 
ahead  to  accurately  maneuver  the  vehicle 

There  is  a  slight  lag  between  the  controls  and  the  robot's  response,  but  with  practice  1 
the  user  can  easily  adapt  to  this. 

The  toggle  method  made  it  harder  to  navigate  and  drive  the  robot.  The  difficulty  was  1 
compounded  by  the  lag  of  the  controller  to  the  robot. 

Sometimes  the  camera  would  get  fuzzy  and  it  was  difficult  to  navigate  during  these  1 
times.  Additionally,  the  lag  time  between  the  GPS  system  and  the  robot  was  quite 
slow  and  one  had  to  constantly  stop  and  wait  for  the  robot's  position  to  catch  up  on 
the  course  making  staying  on  course  somewhat  trying. 

I  found  navigation  of  the  robot  to  be  fairly  easy.  However,  I  experienced  an  error  1 
where  the  robot  would  not  steer  until  it  was  kicked  from  behind.  I  do  not  know 
what  the  problem  was  but  after  the  jolt,  everything  worked  well. 

I  would  rather  have  the  GPS  map  on  screen  so  I  can  glance  at  it  when  navigating.  I  1 

was  often  hesitant  to  do  so  because  I  did  not  want  to  miss  any  of  the  infonnation 
that  I  was  supposed  to  record. 
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Comments 


No.  of  Responses 


It  was  difficult  switching  from  GPS  map  to  the  view  screen  while  staying  on  course.  1 

It  was  hard  to  flip  between  screens  because  it  seemed  like  if  I  flipped  I  would  miss  1 

mission  essential  infonnation. 

It  will  take  time  to  leam  how  to  navigate  the  robot  with  the  controller.  1 

Map  orientation  was  off  track  this  time  when  compared  to  other  previous  tests.  1 

Sometime  robot  will  overheat  which  says  it  is  probably  not  made  for  continuous  use  1 
and  long  hours  of  usage. 

Task  seemed  moderate  in  difficulty,  robot  appeared  to  drift  to  the  right.  I  couldn’t  1 
tell  if  I  was  over  compensating  or  if  it  was  a  function  of  the  robot. 

The  problems  came  with  getting  the  vehicle  around  an  obstacle  or  depth  perception  1 
through  the  camera  view.  Throughout  the  phase  where  the  vehicle  needed  to  go 
around  the  obstacle,  I  found  myself  wishing  there  had  been  a  wide  angle  lens,  or 
side  oriented  camera  to  see  the  obstacle;  and  know  when  to  cut  the  vehicle  back  so 
as  not  to  go  too  far  around  the  obstacle. 

Toggling  back  and  forth  is  a  pain.  1 

C  (Tactile) 

Best  of  the  three.  1 

Easy.  1 

The  belt  made  it  much  easier.  1 

I  thought  the  tactile  belt  made  up  for  the  lack  of  a  map  on  the  screen  at  all  times.  I  1 
always  felt  comfortable  about  where  I  was  and  where  I  was  heading.  I  also  found 
the  objective  easier  with  the  tactile  belt. 

The  belt  system  helped  by  incorporating  a  new  sense  to  the  equation.  It  helped  by  1 
not  overloading  your  vision. 

The  tactile  belt  made  navigating  so  easy  a  caveman  could  do  it.  1 

This  was  much  easier  to  use  than  the  toggle  view.  Instead  of  flipping  screens  I  used  1 

the  camera  view  and  the  belt  to  guide  the  robot  this  way  I  didn’t  miss  any  of  the 
hand  signals. 

Tasks  were  very  good;  the  split  screen  with  belt  combination  would  be  very  helpful  1 
so  you  can  see  simultaneously  the  map  to  associate  yourself  to  way  point  and 
video  of  where  you’re  going  on  ground.  This  I  think  will  enable  you  to  not  get  too 
far  off  your  target  in  relation  to  your  way  point. 

Color  was  out  on  camera.  1 

I  flipped  the  screens  to  get  reoriented  after  getting  around  the  obstacle  and  the  screen  1 
was  flipped  for  me.  I  started  off  in  the  wrong  direction  and  corrected  it.  Close  to 
the  end  the  robot  didn’t  turn  right  for  me  to  get  to  the  finishing  point. 
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Comments 


No.  of  Responses 


The  video  relay  was  a  little  fuzzy  at  times,  but  this  could  have  been  from  being  out  1 
of  reach. 

The  GPS  needs  work,  too  much  lag,  switching  between  view  and  map  is  distracting  1 

2.  Using  the  scale  below,  please  rate  your  ability  to  ability  to  maintain  situation 
awareness  around  the  vehicle  with  the  display  that  you  just  used: 


1 

2 

3 

4 

5 

6 

7 

Extremely  difficult 

Very  difficult 

Difficult 

Neutral 

Easy 

Very  easy 

Extremely  easy 

SITUATION  AWARENESS  TASKS 

MEAN  RESP< 

ONSE 

A 

B 

C 

a.  Awareness  of  terrain 

5.29 

4.54 

5.33 

b.  Awareness  of  hand  signals 

6.13 

5.17 

6.14 

c.  Overall  situation  awareness  of  the  environment  around  vehicle 

5.17 

4.50 

5.33 

A  (Split  Screen) 

Good  views!  1 

The  image  on  the  screen  can  have  better  clarity  and  definition,  but  it  was  effective  1 
enough  to  get  the  job  done. 

Blind  spots  are  still  making  the  situational  awareness  of  the  vehicle  difficult  to  say  1 

the  least. 

Single  camera  made  it  difficult  to  be  aware  of  the  environment  to  the  left  and  right  1 
of  the  vehicle. 

The  camera's  fixed  position  on  the  upward  slope  made  it  harder  to  see  the  terrain  1 


over  the  crest  making  it  a  little  harder  to  see  what  was  coming  next  with  the 
situation  and  test. 

B  (Toggle) 


Clicking  back  and  forth  to  see  where  you  were  on  the  GPS  system  made  it  tricky  to  1 
see  the  hand  signals. 

I  had  to  be  fast  when  I  had  to  toggle  between  the  camera  and  map.  If  I  was  not  fast  1 
then  I  would  have  missed  a  hand  signal. 

It  was  more  difficult  to  observe  hand  signals  because  I  had  to  change  between  1 

navigation  aid  screen  and  driving  camera  screen. 

The  camera  was  having  difficulties  so  at  times  it  was  hard  to  tell  what  hand  signal  1 
was  being  given.  Also  when  I  would  flip  screens  I  would  miss  or  almost  miss  a 
hand  signal. 

Toggling  the  window  brought  about  a  concern  that  I  would  miss  a  hand  signal  as  2 
well  as  not  seeing  where  I  was  going. 

Need  better  visual  field.  1 
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Comments 


No.  of  Responses 


Some  times  picture  quality  will  be  fuzzy  when  going  over  rough,  uneven  terrain.  1 
This  would  be  the  main  downfall  of  the  equipment.  The  single  camera  view,  allows  1 
at  most  60  degrees  of  vision.  The  other  300  degrees  were  completely  black,  and 
could  have  held  any  type  of  enemy.  For  instance,  I  did  not  realize  there  was  a 
group  of  people  walking  with  the  robot,  until  the  end  of  the  exercise.  This  is 
because  they  were  in  the  blind  spot  the  entire  time.  The  terrain  awareness  is 
limited  due  to  the  low  camera  angle.  It  is  mainly  a  depth  perception  problem. 

C  (Tactile) 


Too  easy.  2 

Using  the  tactile  belt,  I  was  able  to  keep  my  eyes  on  the  camera  display  and  only  1 
used  to  GPS  to  confirm  my  waypoint. 

I  came  up  quickly  on  the  ditch  or  gully  that  was  hard  to  see  due  to  the  view  point  or  1 
camera  position.  Other  than  that  it  was  good. 

The  tactile  vibrations  allowed  for  me  to  move  to  the  objective  without  having  to  1 

navigate  using  the  GPS.  It  allowed  me  to  focus  on  the  screen.  With  practice  I 


believe  it  to  be  an  effective  method.  The  vibrations  could  be  different  though,  they 


tend  to  be  slightly  irritating. 

A  guide  identified  a  restriction  on  the  terrain,  but  I  saw  it  before  he  got  in  front  of  1 
the  UGV. 

Situational  awareness  around  the  vehicle  is  still  a  major  concern,  as  the  blind  spots  1 
are  immense. 

Using  only  the  forward  camera  made  it  difficult  to  know  when  the  vehicle  had  1 

passed  the  obstacle. 

You  can  only  be  aware  of  the  situation  around  you  when  you  flip  back  to  the  video,  1 


however  if  you've  flipped  to  map  you  have  no  situational  awareness  when  you  take 
time  to  orient  yourself  to  your  current  position  in  relation  to  your  way  point  for 
those  ten  or  more  seconds  it  may  take  to  do  so. 

3.  Please  check  any  of  the  following  conditions  that  you  may  have  experienced  during  this  trial. 


CONDITION 

No.  of  Responses 

A 

B 

c 

Eyestrain 

1 

1 

0 

Tunnel  vision 

0 

0 

0 

Headaches 

0 

0 

0 

Motion  sickness 

0 

0 

0 

Nausea 

0 

0 

0 

Disorientation 

1 

2 

1 

Dizziness 

0 

0 

0 

Any  other  problems? 

0 

1 

0 
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Comments 


No.  of  Responses 


A  (Split  Screen) 

No  problems.  1 

Slight  disorientation  when  first  starting;  able  to  overcome  after  a  minute.  1 

B  (Toggle) 

Difficult  due  to  visual  field  limitation.  1 

I  was  more  concerned  about  the  time  and  direction  with  having  to  stop  and  orient  the  1 
robot  to  the  way-point. 

Robot  appeared  to  drift  to  the  right.  I  couldn't  tell  if  I  was  over  compensating  or  if  it  1 
was  a  function  of  the  robot. 

Slight  disorientation  when  first  starting  out.  1 

Without  GPS  on  hand,  I  was  not  sure  which  way  I  was  heading  at  times.  1 

C  (Tactile) 

Slight  disorientation  because  one  would  not  be  used  to  it  at  first,  but  as  you  practice  1 
you  become  more  proficient  in  orienting  yourself. 


4.  Using  the  scale  below,  please  rate  the  following  characteristics  (if  applicable)  of  the 
display  that  you  just  used: 


1 

2 

3 

4 

5 

6 

7 

Extremely  bad 

Very  bad 

Bad 

Neutral 

Good 

Very  good 

Extremely  good 

DISPLAY  CHARACTERISTICS 

MEAN 

RESPON 

1SE 

A 

B 

C 

a.  Resolution  (clarity)  of  the  display 

4.44 

4.28 

4.62 

b.  Size  of  objects  appearing  in  the  driving  camera  display 

5.16 

4.88 

4.83 

c.  Size  of  objects  appearing  in  the  map  display 

5.46 

5.12 

5.17 

d.  Comfort  of  viewing  the  driving  display 

5.62 

5.24 

5.21 

e.  Comfort  of  viewing  the  map  display 

5.83 

5.24 

5.46 

f.  Comfort  of  using  the  tactile  display 

- 

- 

5.70 

g.  Strength  of  tactile  display  signal 

- 

- 

5.50 

h.  Contrast  between  objects  on  the  driving  display 

5.08 

5.12 

5.04 

i.  Contrast  between  objects  on  the  map  display 

5.52 

5.20 

5.21 

j.  Driving  display  color 

4.96 

5.08 

4.91 

k.  Map  display  color 

5.68 

5.80 

5.67 

1.  Driving  display  brightness 

5.28 

5.36 

5.46 

m.  Map  display  brightness 

5.64 

5.63 

5.62 

n.  Accuracy  of  map  display 

4.92 

5.00 

4.74 

o.  Accuracy  of  tactile  display  signal 

- 

- 

5.48 

p.  Amount  of  lag  between  vehicle  position  &  driving  camera 

5.20 

5.25 

4.74 

q.  Amount  of  lag  between  vehicle  position  &  navigation  aid  update 

4.08 

3.84 

4.00 

r.  Adequacy  of  display  of  this  type  for  teleoperating  a  robotic 
vehicle 

5.32 

4.32 

5.43 
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Comments 


No.  of  Responses 


A  (Split  Screen) 

This  display  was  the  most  effective  thus  far.  It  was  easy  to  use,  and  provided  the  1 
most  information  in  one  package. 

At  first,  I  did  not  like  the  split  screen,  but  when  I  needed  to  see  the  map  for  1 

navigation  the  page  was  below.  This  is  good  because  an  operator  may  miss  a 
important  camera  event  while  looking  at  the  map  view. 

Resolution  is  not  good.  1 

Black  and  white  screen  made  it  hard  to  distinguish  between  objects.  1 

Video  blurs  sometimes,  GPS  is  slow  to  plot  position.  1 

The  lag  made  the  navigation  uneasy,  but  once  you  understand  the  lag  then  you  can  1 

compensate  for  it.  However,  it  does  require  practice  for  appropriate  compensation. 

I  didn't  like  the  lag  time.  It  would  be  nice  to  have  a  faster  response  with  the  robot.  1 

It  appeared  to  have  the  same  amount  of  GPS  lag  as  with  the  toggle  view.  After  1 

rolling  the  robot,  the  GPS  did  not  work  correctly. 

The  lag  time  really  makes  it  impossible  to  navigate  the  vehicle  based  on  the  1 

navigation  aid  display. 

The  camera  cut  out  on  me  during  a  crucial  moment  as  I  was  attempting  to  maneuver  1 
around  the  obstacle.  The  map  display  is  lagging  a  good  amount  which  takes  some 
getting  used  to.  It  is  still  helpful  in  plotting  a  long  distance  course  but  can  easily 
allow  the  user  to  overcorrect. 

B  (Toggle) 

Overall,  I  think  that  the  operation  of  the  robot  was  good.  1 

Afraid  of  missing  important  hand  signals.  1 

I  think  there  is  a  problem  with  lag  time  on  the  GPS  display.  I  am  not  certain  whether  1 
or  not  that  can  be  corrected.  I  believe  this  could  facilitate  driving  errors  due  to  the 
temporary  inaccuracy  of  the  map  display. 

The  display  had  a  lag  that  would  make  certain  features,  such  as  the  GPS;  combat  1 
ineffective.  The  lag  would  make  you  sit  and  wait,  and  then  the  heading  is 
constantly  adjusting  when  moving  constantly.  If  it  were  a  little  quicker  on  the  lag 
side,  and  then  accounted  for  the  heading  better  it  could  serve  as  a  highly  effective 
tool  for  soldiers. 

Poor  refresh  rate  with  navigation  aid  screen  made  it  impossible  to  navigate  on  this  1 
mode.  Switching  between  screens  made  progress  slow  and  choppy. 

The  toggling  made  it  more  difficult  to  navigate.  I  did  use  the  map  more  to  navigate  1 
in  relation  to  the  actual  terrain  and  way-point.  But,  it  wasn't  as  nice  as  seeing 
where  you  were  going. 

Too  hard  to  see  details.  1 

Video  feed  was  in  and  out  at  times.  1 
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Comments 


No.  of  Responses 


None.  1 

Visual  field  limitation  needs  improvement.  1 

With  no  prior  experience  in  operating  a  robotic  vehicle,  it  is  impossible  for  me  to  1 
judge  the  adequacy  of  the  visual  device  I  used  based  upon  available  devices  in  the 
field  or  on  the  market. 

Would  much  rather  have  the  GPS  on  display  at  all  times.  1 

C  (Tactile) 

I  believe  the  tactile  display  is  the  easiest  to  use.  1 


I  liked  the  tactile  display;  it  was  another  aid  to  reach  the  way-point  and  I  didn’t  have  1 
to  use  distant  objects  to  orient  for  direction  of  travel.  I  could  concentrate  on  what 
was  closer  to  the  robot. 

The  display  is  a  sound  concept  that  needs  only  minor  combat-ready  modifications  to  1 
get  it  to  the  stage  where  it  can  be  tested  in  a  more  realistic  combat  environment. 

The  display  is  good  but  for  missions  >30  minutes  it  could  possibly  cause  eye  strain.  1 


Visual  clarity  and  visual  fields  can  be  improved  with  more  quality  and  fielding.  1 

Camera  definition  could  be  better.  1 

Color  was  out  on  camera  view.  1 

I  think  that  training  on  all  displays  may  be  necessary  depending  on  the  mission.  1 


The  lag  can  be  a  slight  setback  if  you’re  trying  to  turn  within  a  split  second,  because  1 
one  may  tend  to  overturn  if  not  reacting  as  you  would  in  real  time  situational 
awareness. 

The  lag  on  the  navigation  aid  update  made  it  difficult  to  navigate  the  vehicle.  Low  1 
camera  quality  made  it  difficult  to  distinguish  between  terrain  features  (including 
the  obstacle). 

5.  Using  the  scale  below,  what  is  your  overall  rating  of  the  display  that  you  used  this 
iteration? 


1 

2 

3 

4 

5 

6 

7 

Extremely  bad 

Very  bad 

Bad 

Neutral 

Good 

Very  good 

Extremely  good 

MEAN  RESP< 

DNSE 

A 

B 

C 

5.52 

4.44 

5.50 

A  (Split  Screen) 

This  would  be  my  choice  thus  far.  1 

I  was  able  to  read  the  hand  signals  much  better  with  this  display.  1 
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Comments 


No.  of  Responses 


I  used  the  map  very  little  and  attempted  to  use  objects  in  the  distance  in  relation  to  1 
the  map  for  direction  and  navigation.  I  used  the  map  mostly  to  tell  if  I  reached  the 
way-point  and  basic  knowledge  of  direction  and  terrain. 

It  was  a  fairly  good  navigation  activity  given  the  lag  within  the  equipment  and  the  1 
limited  clarity  and  visual  field  from  the  camera.  There  is  a  lot  of  room  for 
improvement  to  remove  the  equipment  lag,  to  improve  the  vision  clarity  and  to 
expand  the  visual  field  from  the  camera.  A  larger  visual  field  would  allow  for 
better  field  bearing  and  navigation. 

Once  the  soldier  is  used  to  map  lag  times,  it  is  not  too  difficult  to  navigate  the  1 

machine.  Improvements  would  include  map  lag  times,  camera  reliability,  and 
camera  resolution. 

B  (Toggle) 

I  believe  that  the  toggle  display  could  be  effectively  used  in  nearly  any  situation  with  1 
the  proper  amount  of  practice. 

Good  system,  it  will  take  longer  to  train  on  but  can  be  as  effective  as  split  screen  if  1 
soldier  used  map  strictly  for  navigation  due  to  slow  GPS  plotting. 

Overall,  I  would  say  the  system  is  a  sound  idea.  There  are  a  few  things  that  need  to  1 
be  worked  out  to  make  the  device  combat  ready.  The  main  things  being  a  better 
sense  of  depth  perception  on  the  video  display,  possibly  achieved  through  raising 
the  camera  height;  lowering  the  lag  between  GPS  and  actual  position,  and  then  I 
would  also  recommend  calibrating  the  system  to  be  used  in  a  stressful 
environment.  We  used  it  while  our  bodies  are  at  ease,  and  the  joystick/system 
required  a  very  sensitive  touch. 

If  the  lag  time  could  be  fixed  the  display  was  pretty  good.  The  camera  fuzziness  1 
made  it  difficult  to  see  terrain  features  such  as  rocks,  bumps  in  the  road,  etc. 

Hitting  these  features  also  caused  the  camera  to  be  fuzzy  so  it  would  help  to  see 


the  features  better  prior  to  hitting  them. 

It  was  harder  to  navigate  by  not  seeing  or  feeling  and  view  what  was  actually  in  1 

front  of  you.  It  limited  the  ability  to  accurately  travel  and  operate  the  robot. 
Overheating  may  become  problematic  if  continuous  usage  is  required.  1 

Sometimes  the  image  will  be  distorted  with  slight  green  streaks.  1 

Toggle  was  less  convenient  than  the  split  due  to  the  flipping  back  and  forth.  1 

C  (Tactile) 

My  favorite  of  the  three.  1 

Too  easy.  1 

This  so  far  this  has  been  the  best  system  as  I  do  not  have  to  take  my  eyes  off  the  1 

driving  camera  in  order  to  navigate  to  my  objective.  An  effective  system  that  is 
also  helpful  when  finding  exact  points  such  as  the  objective. 

Belt  is  easy  to  use  and  much  more  accurate  than  the  navigation  alone.  1 

Overall  a  good  system  to  use  on  any  maneuver/search  operation.  1 
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Comments 


No.  of  Responses 


This  setup  was  my  2nd  favorite  choice.  I  think  that  the  additional  sense  added,  1 

helped  the  user;  by  not  allowing  him  to  overload  one  particular  sense.  In  this  case, 
the  sense  of  sight  was  complimented  by  a  sense  of  touch.  The  combat 
effectiveness  of  this  approach  could  prove  problematic  though.  The  average 
soldier  in  combat  would  have  to  have  the  ability  to  distinguish  between  the  signals 
from  the  belt,  and  then  other  outside  factors  of  touch,  i.e.,  something  hitting  them, 
overcoming  the  adrenaline  factor. 

This  was  an  very  effective  method  of  controlling  the  machine.  The  belt  was  slightly  1 
irritating  though,  but  the  thought  behind  such  a  belt  makes  a  lot  of  sense. 

The  vibrations  are  only  helpful  if  you  concentrate  on  the  vibrations,  it's  easier  to  1 
follow  camera/map. 


Gianoros  Motion  Sickness  Question; 

6.  Using  the  scale  below,  please  rate  how  accurately  the  following  statements  describe 
your  current  state. 


123456789 
Not  at  all  < - >  Severely 


MEAN  RESPO] 

\SE 

A 

B 

C 

I  felt  sick  to  my  stomach 

1.04 

1.13 

1.00 

I  felt  disoriented 

1.16 

1.13 

1.12 

I  felt  faint-like 

1.04 

1.00 

1.00 

I  felt  tired/fatigued 

1.24 

1.33 

1.40 

I  felt  annoyed/irritated 

1.20 

1.50 

1.64 

I  felt  nauseated 

1.04 

1.00 

1.00 

I  felt  sweaty 

1.64 

1.71 

1.68 

I  felt  hot/warm 

2.24 

1.87 

2.08 

I  felt  queasy 

1.04 

1.00 

1.00 

I  felt  dizzy 

1.04 

1.00 

1.00 

I  felt  lightheaded 

1.04 

1.00 

1.00 

I  felt  like  I  was  spinning 

1.04 

1.00 

1.00 

I  felt  drowsy 

1.08 

1.21 

1.16 

I  felt  as  if  I  may  vomit 

1.00 

1.00 

1.00 

I  felt  clammy/cold  sweat 

1.00 

1.00 

1.00 

I  felt  uneasy 

1.08 

1.09 

1.08 
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END  OF  EXPERIMENT 


SAMPLE  SIZE  =  25 

1.  Using  the  scale  below,  please  rate  the  training  that  you  received  in  the  following  areas: 


1 

2 

3 

4 

5 

6 

7 

Extremely  bad 

Very  bad 

Bad 

Neutral 

Good 

Very  good 

Extremely  good 

MEAN  RESPONSE 

a.  How  to  drive  the  robot 

6.04 

b.  Time  provided  to  practice  driving  the  robot 

5.96 

c.  How  to  complete  Leg  A  (obstacle  negotiation)  of  the  course 

5.92 

d.  Evaluation  of  the  practice  lane 

5.92 

e.  How  to  use  the  navigation  map 

6.00 

f.  How  to  use  the  tactile  belt 

6.17 

g.  How  well  you  expected  to  perform  on  the  actual  course  after  training 

5.80 

h.  Overall  evaluation  of  the  training  course 

6.24 

2.  What  were  the  easiest  and  hardest  training  tasks  to  learn? 

Comments  No.  of  Responses 


Easiest 

How  to  operate  the  robot.  5 

The  whole  thing  was  pretty  easy.  1 

The  actual  task.  1 

Moving  the  vehicle  forward  and  backward  in  any  given  direction.  1 

Basic  movement.  1 

Clear  concise  instructions  made  the  task  of  operating  the  robot  easy.  2 

Learning  the  control  functions.  1 

The  easiest  training  tasks  were  orienting  to  the  map  and  camera  feed.  1 

Even  though  I  did  well  in  training,  I  was  still  a  bit  apprehensive  about  the  actual  1 

course.  After  I  started  it  though  I  regained  my  confidence. 

Tactile  belt.  3 

Negotiating  the  obstacle  course  using  the  tactile  belt.  2 

Familiarization  with  the  use  of  the  map  and  tactile  belt.  1 

Negotiation  and  navigation  of  the  course  and  time.  1 

Learning  how  to  use  the  controller  with  the  robot  and  the  time  delay.  1 

Operation  of  the  viewing  screen.  1 

Split  screen  and  Belt.  1 
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Comments 


No.  of  Responses 


Understanding  the  navigation  map.  It  was  easy  to  manipulate  the  course  via  the  1 

navigation  system. 

Viewing  direction  via  the  monitor  from  the  robot's  perspective.  1 

Hardest 

The  GPS  lag  was  pretty  bad  in  places.  2 

The  lag  on  the  response  time  when  controlling  the  robot  made  this  course  more  2 

difficult  than  it  should  be. 

The  lag  time  and  orienting  the  robot.  1 

The  camera  needs  some  work.  1 

Beginning.  1 

Control  and  maneuvering  the  robot,  i.e.,  trying  not  to  over  compensate  your  turns.  1 

Controlling  the  robot  with  the  joystick  and  negotiating  the  terrain.  1 

The  most  difficult  thing  was  getting  the  hang  of  how  much  pressure  to  put  on  the  joy  1 
stick  to  get  it  to  move  in  the  direction  you  want  it  to  go. 

Using  the  joystick  at  first,  after  training  it  was  easy.  1 

Driving,  because  the  controls  can  be  touchy  sometimes  and  non-responsive  at  others.  1 
Learning  how  to  turn  the  robot  around  corners  when  moving.  1 

Learning  the  feel  from  the  delay  on  the  controller  versus  the  response  of  the  robot.  1 

Judge  the  distance  to  object  and  navigate  around  object  outside  of  camera  view.  1 

Judging  distances  from  the  monitor.  1 

Keep  speed  slow.  1 

Turning  the  robot  at  higher  speeds.  1 

Navigation  around  objects.  2 

Negotiating  the  obstacles.  1 

The  most  difficult  task  was  negotiating  the  obstacle  course  using  the  flip  screen.  2 
Speed  and  turning  control,  adjusting  for  the  weaker  resolution  of  the  driving  camera,  1 
and  flipping  back  and  forth  between  the  camera  and  the  map  while  navigating  and 
attempting  to  remember  the  infonnation  using  the  "toggle"  method. 

Steering  the  robot  took  some  getting  used  to.  1 

Steering  the  vehicle  left  and  right  without  overcompensating.  1 
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3.  What  are  your  comments  on  the  training  course? 

Comments  No.  of  Responses 


Felt  comfortable  driving  the  robot  after  only  fifteen  minutes  of  initial  training.  1 

Good  course,  technical  difficulties  were  numerous  although  staff  did  a  good  job  of  1 

working  through  them. 

Good.  2 

I  think  training  was  well  organized  and  that  every  trainer  was  trained  themselves  and  1 
capable. 

More  time  on  the  obstacle  negotiations  would  be  beneficial.  1 


The  course  was  set  up  well  and  enjoyed  practicing  with  the  robot  and  appreciate  the  1 
opportunity. 

The  lengthy  breaks  in  between  operator  rotations  gives  the  operator  sufficient  time  1 
to  rest  eyes. 

The  robotic  training  course  was  very  interesting  and  will  be  beneficial  to  me  in  my  1 
military  career. 

The  staff  was  professional  and  thorough  in  their  briefings  and  training.  The  1 

expectations  were  clearly  expressed  and  all  participants  were  given  the  same 
treatment  and  scenarios. 

The  training  course  was  excellent  and  very  infonnative  in  future  technology  1 

Overall  the  training  was  great,  but  I  feel  that  more  than  one  Robot  should  be  1 

available  to  continue  training.  When  the  Robot  overheats  due  to  mechanical 
failure  there  should  be  more  time  allotted,  even  though  the  time  started  once  you 
restarted  over  again.  You  should  always  have  second  course  of  action  if 
mechanical  failures  with  equipment  take  place. 

The  training  should  have  included  all  the  types  of  terrain  the  robot  should  be  able  to  1 
operate. 

Need  a  wider  area  to  romp  and  gain  a  feel  for  the  robot.  1 

The  turnaround  point  should  be  more  visible.  1 

4.  Please  rank  order  the  displays  in  the  order  of  your  preference  -  with  1  being  your 

favorite,  2  your  second  choice,  etc. 


Display 

#  preferred 

% 

preferred 

A  (split  screen) 

39 

52% 

B  (toggle) 

7 

7% 

C  (tactile) 

31 

41% 

Sum 

75 

100% 
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5.  Do  you  have  any  suggestions  for  ways  to  increase  the  effectiveness  of  the  displays? 


Comments  No.  of  Responses 

Display  A  (split  screen) 

I  thought  it  was  easy  to  see  both  screens  at  the  same  time.  1 

It  was  good.  1 

Having  the  screen  on  the  right  instead  of  the  bottom.  1 

I  would  move  my  eyes  around  less  by  using  tactile  with  the  map,  i.e.,  in  navigating  1 

in  woods  (or  open  desert)  with  everything  seeming  the  same.  The  tactile  would 
point  me  in  the  right  direction  and  my  eyes  would  remain  on  the  view  screen.  It  is 


another  layer  to  navigating  the  robot. 

If  at  all  possible,  try  and  work  out  some  of  the  delay  in  the  video,  whether  it  be  from  1 
power  output  or  whatever  requirements  it  might  take. 

If  the  GPS  lag  can  be  lessened  the  course  would  be  easier.  3 

Increase  the  brightness  of  the  screen.  1 

Increase  the  resolution  of  the  main  screen.  It  is  hard  to  view  the  course  depending  2 
on  what  part  of  the  day  it  is. 

Video  with  higher  resolution.  Bigger  Image  on  the  screen.  1 

Adjust  the  controls  to  compensate  for  the  "touchiness".  1 

Make  it  bigger;  the  camera  display  is  too  small  to  see  terrain  details.  1 

Make  the  camera  spin  for  corners  and  turns.  1 

Being  able  to  make  video  map  more  stable.  1 

Why  not  add  the  tactile  belt?  1 

Would  like  to  flip  the  screens  so  map  point  is  on  top.  1 

Display  B  (toggle) 

Did  not  like  this  one  at  all  -  very  difficult  to  see  hand  signals,  navigate  and  see  your  1 
position  on  the  map  with  the  lag  time  factor. 

Did  not  like  this  system  at  all.  Felt  disorientated  and  slow.  1 

Don't  use  it.  One  can  easily  miss  hand  signals  using  this  display.  1 

Eliminate  this  method,  having  to  toggle  takes  more  time,  senses,  and  it  temporarily  1 
takes  the  solider  out  of  the  fight.  That  split  second  could  mean  life  or  death. 

Have  both  screens.  It  added  to  frustration  and  the  time  of  the  navigation.  1 

Having  the  switch  on  the  key  instead  of  the  pen.  1 

I  don’t  really  know  what  to  say  for  improvement  but  this  method  seemed  to  be  the  1 
least  effective  because  you  miss  things  jumping  between  screens. 


I  really  did  not  like  the  flip  screen.  It  took  away  from  concentration  as  well  as  waste  1 


valuable  time  flipping  back  and  forth  between  screens. 

It  was  time  consuming  flipping  to  both  screens.  1 

Make  a  secondary  toggle  switch  on  the  controller  (joystick).  2 

Maybe  make  the  toggle  some  sort  of  switch  you  can  hold  in  your  hand.  1 

Being  able  to  make  video  map  more  stable.  1 

Place  arrow  in  corner  of  screen,  arrow  pointing  toward  objective.  1 
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Probably  just  don't  use  the  toggle.  Takes  a  moment  to  shift  back  and  forth  and  gain  a  1 
good  understanding  of  navigation,  could  pose  a  hazard  in  a  combat  zone  to 
soldiers. 

Video  with  higher  resolution.  Bigger  Image  on  the  screen.  1 

The  flip  screen  made  it  impossible  to  stay  situationally  aware  at  all  times.  If  the  1 

processor  were  fast  enough  to  allow  for  instant  flipping  it  could  work.  As  far  as 
finger  flashing  in  front  of  the  camera  though,  they  still  would  be  a  high  chance  that 
they  would  be  missed.  If  you  had  the  ability  to  record  every  time  you  flip  the 
screen  then  you  would  know  that  you  would  not  be  missing  anything,  however  the 
cost  of  such  an  option  is  something  that  I  am  not  aware  of. 

The  switching/flipping  back  and  forth  distracts  from  mission  perfonnance  and  1 

accomplishment. 

They  both  can  be  effective  if  trained  correctly.  The  toggle  can  be  built  into  a  smaller  1 
unit  and  would  be  lighter.  Make  the  GPS  corrections  quicker. 

Display  C  (tactile) 

Excellent.  1 

As  with  each  display,  It  seemed  that  the  camera  had  trouble  with  it's  resolution  over  1 
rough  ground.  It  seems  as  though  the  pixels  were  blinking  out.  The  problem 
corrected  itself  in  a  second  or  so. 

It  was  still  harder  to  know  that  you  reached  the  way-point  even  with  the  signals.  I  1 
got  the  signal  but  with  just  once  I  still  questioned  if  I  missed  it  being  my  first  time. 

Lag  time  creates  inefficiencies.  1 

Make  the  electrode  pads  slightly  larger  to  compensate  for  larger  operators.  1 

Being  able  to  make  video  map  more  stable.  1 

Place  arrow  in  corner  of  screen,  arrow  pointing  toward  objective.  1 

Possibly  having  the  split  screen  and  the  tactile  method  together  would  be  a  really  1 

efficient  way  of  accomplishing  the  task.  That  way  the  sensors  can  guide  you  but 
you  also  have  the  map  readily  accessible  to  look  at  if  you  feel  like  you  are  getting 
off  course. 

Possibly  integrate  it  with  the  split-screen.  The  only  negative  aspect  was  that  you  1 
still  had  to  toggle  to  the  GPS. 

The  6  o’clock  signal  was  sometimes  faint  because  of  the  shape  of  my  back.  This  1 

signal  should  be  made  stronger. 

The  belt  could  produce  a  higher  output  for  a  better  feel  of  direction  and  it  needs  a  1 
faster  return  of  data  so  operator  does  not  get  confused  on  actual  direction  needing 
to  go. 

Video  with  higher  resolution.  Bigger  Image  on  the  screen.  1 

The  camera  display  could  be  larger.  1 

The  problem  I  had  with  Tactile  is  that  you  needed  to  switch  over  to  view  the  map.  1 
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The  tactile  display  may  be  more  efficient  if  the  belt  were  used  with  the  split  screen  1 
display. 

Why  not  add  the  split  screen?  I  know  the  object  of  this  research  is  to  create  the  most  1 
compact  and  mobile  system,  but  considering  you  are  bringing  a  large  robot  with 
you,  what  difference  does  another  few  inches  of  a  screen  with  (hopefully)  better 
resolution  make. 

The  tactile  type  was  irritating  at  times  because  of  the  vibrations  that  were  constantly  1 
present.  It  is  a  good  system  but  the  intensity  was  possibly  too  much  for  just  putting 
on  the  screen. 

Get  rid  of  it.  1 

The  vibrations  are  only  helpful  if  other  screens  are  unavailable.  1 

6.  Please  provide  suggestions  on  ways  to  improve  the  driving  lanes  that  you  negotiated? 

I  thought  the  lanes  were  set  up  well.  2 

I  believe  the  terrain  for  the  lanes  were  ok.  Driving  lanes  should  never  be  perfect.  1 

They  were  pretty  good  for  reality  wise  because  you  never  know  what  terrain  you  1 

will  come  up  against.  So  practicing  in  a  more  rough  terrain  is  probably  the  best 
way. 

Perhaps  more  (smaller  but  impassable)  obstructions  could  be  used.  1 

Add  more  obstacles.  1 

Color  bushes  and  other  objects  that  may  harm  the  robot  so  navigation  can  be  more  1 
accurate  around  obstacles  instead  of  just  running  them  over. 

Don't  run  the  lanes  in  the  rain  unless  robot  is  further  developed  to  handle  wet  1 

conditions  to  prevent  inaction  during  handling. 

If  the  course  requires  that  a  soldier  stand  in  front  of  ravines  in  order  to  tell  you  when  1 
to  halt  the  robot,  that  should  tell  us  that  the  camera  display  isn’t  picking  up 
sufficient  terrain  detail  for  real-world  use. 

Improve  on  stabilizing  the  camera  better  while  the  robot  travels  over  various  terrain.  1 

Making  sure  that  the  paths  are  clear  from  major  obstacles.  Ran  into  a  dead  tree  in  1 

the  middle  of  one  of  the  runs. 

More  variation  other  than  the  standard  engineering  tape  and  concertina  wire.  Maybe  1 
make  a  makeshift  wall  or  two  to  simulate  driving  around  buildings. 

One  test  should  be  done  on  smooth  or  pavement  area  to  see  the  full  capacity  of  the  1 
robot. 

Possibly  make  the  engineering  tape  higher  up.  This  would  make  picking  a  point  and  1 
driving  towards  it  easier,  as  you  can  see  the  lane  further  out. 
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The  tents  that  were  provided  were  not  very  strong  in  the  conditions  here  at  Fort  1 

Benning.  If  better  tents  were  provided  it  could  have  helped  in  the  process.  The 
wind  was  extremely  rough  today. 

Use  bright  orange  tape.  1 

Use  red  tape  to  identify  obstacles,  as  the  yellow  tape  was  difficult  to  see  because  of  1 
poor  screen  resolution. 

With  the  taller  grass  and  brush  it  was  sometimes  hard  to  see  the  lines  they  came  up  1 
quickly  sometimes.  I  had  to  try  and  drive  more  carefully. 

7.  What  are  your  comments  on  infonnation  needed  for  teleoperating  a  vehicle? 

A  truly  remarkable  system  that  is  very  user  friendly.  Some  navigation  systems  are  1 
definitely  easier  than  others  as  noted  previously. 

Good.  1 

Operators  must  be  given  a  driving  course  that  focuses  on  compensating  for  on  left  /  1 

right  steering  delay. 

The  vehicle  overheats  often.  1 
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Acronyms 


2-D 

two-dimensional 

3-D 

three-dimensional 

ANOVA 

analysis  of  variance 

ARL 

U.S.  Army  Research  Laboratory 

GPS 

global  positioning  system 

MRT 

Multiple  Resource  Theory 

NASA 

National  Aeronautics  and  Space  Administration 

ocs 

Officer  Candidate  School 

ocu 

operator  control  units 

SA 

situation  awareness 

TARDEC 

Tank  and  Automotive  Research,  Development  and  Engineering  Center 

TLX 

Task  Load  Index 

UAV 

unmanned  aerial  vehicle 

UCF 

University  of  Central  Florida 

XGA 

extended  graphics  array 
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1  DEFENSE  TECHNICAL 
(PDF  INFORMATION  CTR 
only)  DTIC  OCA 

8725  JOHN  J  KINGMAN  RD 
STE  0944 

FORT  BELVOIR  VA  22060-6218 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
IMNE  ALC  IMS 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
AMSRD  ARL  Cl  OK  TL 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
AMSRD  ARL  Cl  OK  PE 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  ML  J  MARTIN 
MYER  CENTER  RM  2D3 1 1 
FT  MONMOUTH  NJ  07703-5601 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MZ  A  DAVISON 
320  MANSCEN  LOOP  STE  115 
FT  LEONARD  WOOD  MO  65473 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MD  T  DAVIS 
BLDG  5400  RM  C242 
REDSTONE  ARSENAL  AL  35898-7290 

1  COMMANDANT  USAADASCH 
ATSA  CD 

AMSRD  ARL  HR  ME  DR  HAWLEY 
5800  CARTER  RD 
FT  BLISS  TX  79916-3802 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MM  DR  V  J  RICE 
BLDG  4011  RM  217 
1750  GREELEY  RD 
FT  SAM  HOUSTON  TX  78234-5002 


1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MG  R  SPINE 
BLDG  333 

PICATINNY  ARSENAL  NJ  07806-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 
ARMC  FIELD  ELEMENTT 
AMSRD  ARL  HR  MH  C  BURNS 
THIRD  AVE  BLDG  1467B  RM  336 
FT  KNOX  KY  40121 

1  ARMY  RSCH  LABORATORY  -  HRED 
AWC  FIELD  ELEMENT 
AMSRD  ARL  HR  MJ  D  DURBIN 
BLDG  4506  (DCD)  RM  107 
FT  RUCKER  AL  36362-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MK  MR  J  REINHART 
10125  KINGMAN  RD 
FT  BELVOIR  VA  22060-5828 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MY  M  BARNES 
2520  HEALY  AVE 
STE  1172  BLDG  51005 
FT  HUACHUCA  AZ  85613-7069 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MP  D  UNGV ARSKY 
POPE  HALL  BLDG  4709 
BCBL  806  HARRISON  DR 
FT  LEAVENWORTH  KS  66027-2302 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MJF  J  HANSBERGER 
JFCOM  JOINT  EXPERIMENTATION  J9 
JOINT  FUTURES  LAB 
115  LAKEVIEW  PKWY  STE  B 
SUFFOLK  VA  23435 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MQ  M  R  FLETCHER 
NATICK  SOLDIER  CTR 
AMSRD  NSC  WSE  BLDG  3  RM  343 
NATICK  MA  01760-5020 

1  ARMY  RSCH  LABORATORY-HRED 
AMSRD  ARL  HR  MT  J  CHEN 
12423  RESEARCH  PKWY 
ORLANDO  FL  32826 
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ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MT  C  KORTENHAUS 
12350  RESEARCH  PKWY 
ORLANDO  FL  32826 

ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MS  C  MANASCO 
SIGNAL  TOWERS 
BLDG  29808A  RM  303 
FT  GORDON  GA  30905-5233 

ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MU 
6501  E  11  MILE  RD  MS  284 
BLDG  200A  2ND  FL  RM  2104 
WARREN  MI  48397-5000 

ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MF  C  HERNANDEZ 
2421  NW  AUSTIN  RD  STE  220 
FT  SILL  OK  73503-9042 

ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MV 
S  MIDDLEBROOK 
91012  STATION  AVE  RM  348 
FT  HOOD  TX  76544-5073 

ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MN  R  SPENCER 
DCSFDI  HF 

HQ  USASOC  BLDG  E2929 
FT  BRAGG  NC  28310-5000 

ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MW  E  REDDEN 
BLDG  4  ROOM  332 
FT  BENNING  GA  31905-5400 

ARMY  G1 

DAPE  MR  B  KNAPP 

300  ARMY  PENTAGON  RM  2C489 

WASHINGTON  DC  20310-0300 
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ABERDEEN  PROVING  GROUND 
1  DIR  USARL 

AMSRD  ARL  Cl  OK  TP  (BLDG  4600) 

5  DIR  USARL 

AMSRD  ARL  Cl  OK  TP 
S  FOPPIANO 
AMSRD  ARL  HR 
T  LETOWSKI 
AMSRD  ARL  HR  MD 
D  HARRAH 
AMSRD  ARL  HR  MR 
F  PARAGALLO 
AMSRD  ARL  HR  SD 
B  AMREIN 

TOTAL:  33  (1  ELEC,  1  CD,  31  HCs) 


US  ARMY  RSRCH  DEV  &  ENGR 
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SYS  OF  SYS  INTEGRATION 
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6000  6TH  ST  STE  100 
FT  BELVOIR  VA  22060-5608 
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